Clothing design method and device based on bidirectional feature fusion of sketch and texture map
By performing multiple rounds of feature fusion of sketches and texture diagrams, and inputting the clothing generation model with the original image, the problem of poor fusion of sketches and texture diagrams in the existing technology is solved, and the generation quality and reliability of clothing design drawings are improved.
Patent Information
- Application Number
- CN202411680118.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, when using sketches and textures for clothing design, it is difficult to effectively integrate the feature details of both, resulting in a lower quality of the generated clothing image.
By fusion of multiple features of the sketch and texture map of the design clothing, multiple fusion feature pairs are obtained, and the results after the fusion of these features and the original sketch and texture map are input to the first clothing generation model, and generated based on the preset guidance scale.
Effectively integrate the feature details of sketches and texture drawings to improve the reliability and quality of clothing design drawing generation.
Smart Images
Figure CN119598538A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of clothing design, and in particular to a clothing design method and device based on bidirectional feature fusion of sketches and texture images. Background Art
[0002] In the clothing design industry, traditional clothing design usually requires experienced designers with artistic aesthetic ability. This method of purely relying on professional designers has the problems of high design threshold and high time cost. To solve this problem, the technology of using artificial intelligence generated content (AIGC) technology for clothing design has been continuously developed in recent years.
[0003] Generally speaking, clothing design based on AIGC technology can be summarized as an image-to-image (I2I) translation problem. There are currently three main models and methods that can be used for clothing design in the prior art, namely, Generative Adversarial Network (GAN), Vision Transformers (VIT) model and Latent Diffusion Model (LDM). When the semantic gap between the given sketch and texture image is too large, the structure of GAN is difficult to decouple the image structure and style features, resulting in unsatisfactory quality of generated clothing images; VIT generates clothing images by transferring the appearance of the reference image to the source image, but this method often lacks pre-training data, and when the semantic gap between the source image and the reference image is large, the texture details cannot be effectively transferred, resulting in poor quality of generated clothing images; LDM has been applied to generate clothing images by simply splicing the sketch and texture image as LDM input, but simple splicing cannot effectively fuse the feature details of the two, resulting in room for improvement in the quality of generated clothing images. Based on the above technical background, how to improve the quality of generated clothing images when sketches and texture maps are used as input is still a technical problem that needs to be urgently solved in the existing technology. Summary of the invention
[0004] The present application provides a clothing design method and device based on bidirectional feature fusion of sketches and texture images to solve the technical problem of low quality of clothing images generated by the prior art.
[0005] According to a first aspect of the implementation of the present application, a clothing design method based on bidirectional feature fusion of sketch and texture map is provided, comprising:
[0006] Performing multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed to obtain multiple fused feature pairs; wherein the sketch features and the texture map features are generated based on the sketch and texture map of the clothing to be designed, respectively; in each round of feature fusion, adjusting the input sketch features based on the input texture map features to obtain output sketch features, and adjusting the input texture map features based on the input sketch features to obtain output texture map features, and taking the combination of the output sketch features and the output texture map features as the output fused feature pair;
[0007] The sketch, the texture map and the multiple fusion feature pairs are input into a first clothing generation model, and generated based on a preset guiding scale to obtain a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
[0008] The present application first performs multiple rounds of feature fusion on the sketch and texture map of the clothing to be designed. In each round of feature fusion, the sketch and texture map are fused with each other to adjust each other's feature output, and multiple fused feature pairs are obtained. The feature details of the sketch and texture map can be effectively fused, which is beneficial to the migration of texture details on the sketch during subsequent clothing generation. The sketch, texture map and multiple fused feature pairs are then input into the first clothing generation model, and generation is performed based on a preset guidance scale. This can stabilize the clothing generation scale and improve the reliability of clothing design drawing generation.
[0009] In certain embodiments of the present application, the step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes:
[0010] Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features;
[0011] Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair;
[0012] The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
[0013] The present application first downsamples the sketch features and the texture map features to obtain sampled sketch features and sampled texture map features, respectively, and then performs parameter extraction to obtain sketch parameter pairs and texture map parameter pairs, respectively, so as to obtain parameters that can adjust the output of each other, so that when the sampled sketch features are adjusted according to the texture map parameter pairs and the sampled texture map features are adjusted according to the sketch parameter pairs, the feature details of the sketch and the texture map are effectively fused, thereby improving the degree of mutual fusion between the sketch and the texture map.
[0014] In certain embodiments of the present application, the step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain the output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain the output texture map feature specifically includes:
[0015] The adjusting of the sampling sketch feature is specifically as follows:
[0016]
[0017] Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively;
[0018] The adjusting of the sampling texture map feature is specifically:
[0019]
[0020] Among them, f′ i To output texture features, is a pair of sketch parameters, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
[0021] This application is based on feature fusion rounds, adjusts the sampled sketch features according to texture map parameters and adjusts the sampled texture map features according to sketch parameters, which can effectively fuse the feature details of the sketch and texture map and improve the degree of mutual fusion between the sketch and texture map.
[0022] In certain embodiments of the present application, the first clothing generation model is obtained by training based on a preset clothing generation model, specifically including:
[0023] Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map;
[0024] The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
[0025] This application first obtains clothing image training data, and then inputs a preset clothing generation model for iterative training. During iteration, the model parameters are optimized through the overall generation loss composed of noise prediction loss, variational lower bound loss and style loss, which can enable the model to converge quickly to the task requirements and be more in line with the actual task of clothing generation.
[0026] In certain embodiments of the present application, each round of iteration optimizes the model parameters based on the overall generation loss, specifically including:
[0027] In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs;
[0028] Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise;
[0029] De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image;
[0030] Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph;
[0031] The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
[0032] In each round of iteration, the present application first adds noise to the clothing design drawing based on the preset noise to obtain a clothing noise map, and performs multiple rounds of feature fusion on the training sketch and the training texture map to obtain multiple training fusion feature pairs, so as to obtain basic training data, and then predicts the clothing noise map to obtain predicted noise, and calculates the noise prediction loss by combining the multiple training fusion feature pairs and the preset noise, so as to reasonably calculate the noise prediction loss based on the actual needs of the task; then, the clothing noise map is denoised based on the predicted noise to obtain a clothing prediction map, and then the style loss and the variational lower bound loss are calculated, so as to reasonably calculate the style loss and the variational lower bound loss based on the actual needs of the task; thus, when the noise prediction loss, the style loss and the variational lower bound loss are combined to obtain the overall generation loss and optimize the model parameters of the preset clothing generation model, it is more in line with the actual task of clothing generation.
[0033] In certain embodiments of the present application, the overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including:
[0034] L=L simple +L υlb +Lstyle
[0035]
[0036] Among them, L, L simple , L υlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,..., T is the time step, T is the sequence length, z0=E(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
[0037] In certain embodiments of the present application, the step of inputting the sketch, the texture map, and the plurality of fusion feature pairs into a first clothing generation model, generating the clothing design drawing of the clothing to be designed based on a preset guidance scale, specifically includes:
[0038] Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing;
[0039] The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows:
[0040]
[0041] in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ (·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S and O are sketch and texture map respectively, S s , Os To set the guideline scale;
[0042] The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.
[0043] The present application first inputs a sketch, a texture map and a plurality of fusion feature pairs into a first clothing generation model, and gradually performs denoising based on a preset guiding scale to generate a predicted clothing design drawing, which can stabilize the clothing generation scale, thereby improving the reliability of clothing design drawing generation when decoding the predicted clothing design drawing to obtain the clothing design drawing of the clothing to be designed.
[0044] According to a second aspect of the implementation of the present application, there is provided a clothing design device based on bidirectional feature fusion of sketches and texture maps, comprising a feature fusion module and a clothing generation module;
[0045] The feature fusion module is used to perform multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed to obtain multiple fused feature pairs; wherein the sketch features and the texture map features are generated based on the sketch and texture map of the clothing to be designed respectively; in each round of feature fusion, the input sketch features are adjusted based on the input texture map features to obtain output sketch features, and the input texture map features are adjusted based on the input sketch features to obtain output texture map features, and the combination of the output sketch features and the output texture map features is used as the output fused feature pair;
[0046] The clothing generation module is used to input the sketch, the texture map and the multiple fusion feature pairs into a first clothing generation model, generate based on a preset guidance scale, and obtain a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
[0047] In certain embodiments of the present application, the step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes:
[0048] Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features;
[0049] Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair;
[0050] The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
[0051] In certain embodiments of the present application, the step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain the output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain the output texture map feature specifically includes:
[0052] The adjusting of the sampling sketch feature is specifically as follows:
[0053]
[0054] Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively;
[0055] The adjusting of the sampling texture map feature is specifically:
[0056]
[0057] Among them, f′ i To output texture features, is a pair of sketch parameters, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
[0058] In certain embodiments of the present application, the first clothing generation model is obtained by training based on a preset clothing generation model, specifically including:
[0059] Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map;
[0060] The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
[0061] In certain embodiments of the present application, each round of iteration optimizes the model parameters based on the overall generation loss, specifically including:
[0062] In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs;
[0063] Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise;
[0064] De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image;
[0065] Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph;
[0066] The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
[0067] In certain embodiments of the present application, the overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including:
[0068] L=L simple +L υlb +L style
[0069]
[0070] Among them, L, L simple , L υlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,.., T is the time step, T is the sequence length, z0=E(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
[0071] In certain embodiments of the present application, the step of inputting the sketch, the texture map, and the plurality of fusion feature pairs into a first clothing generation model, generating the clothing design drawing of the clothing to be designed based on a preset guidance scale, specifically includes:
[0072] Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing;
[0073] The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows:
[0074]
[0075] in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ (·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S and O are sketch and texture map respectively, S s , O s To set the guideline scale;
[0076] The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.
[0077] The present application first performs multiple rounds of feature fusion on the sketch and texture map of the clothing to be designed. In each round of feature fusion, the sketch and texture map are fused with each other to adjust each other's feature output, and multiple fused feature pairs are obtained. The feature details of the sketch and texture map can be effectively fused, which is beneficial to the migration of texture details on the sketch during subsequent clothing generation. The sketch, texture map and multiple fused feature pairs are then input into the first clothing generation model, and generation is performed based on a preset guidance scale. This can stabilize the clothing generation scale and improve the reliability of clothing design drawing generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0078] Figure 1 : A schematic diagram of a flow chart of a clothing design method based on bidirectional feature fusion of sketch and texture map shown in certain embodiments of the present application;
[0079] Figure 2 : A module structure diagram of a clothing design device based on bidirectional feature fusion of sketch and texture map shown in some embodiments of the present application;
[0080] Figure 3 : A bidirectional feature fusion network framework diagram shown in certain embodiments of the present application;
[0081] Figure 4: A clothing design model framework structure diagram shown in certain embodiments of the present application. DETAILED DESCRIPTION
[0082] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0083] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.
[0084] There are three main models and methods that can be used for clothing design in the prior art, namely GAN, VIT and LDM. When the semantic gap between the sketch and the texture map is too large, GAN is difficult to decouple the image structure and style features, resulting in low quality of clothing image generation. VIT is often insufficient in pre-training data, and when the semantic gap between the sketch and the texture map is too large, it is difficult to effectively transfer the texture details, which also leads to low quality of clothing image generation. LDM has been applied in clothing design to generate clothing images by simply splicing sketches and texture maps into LDM, but simple splicing cannot effectively integrate the characteristic details of sketches and texture maps, resulting in the quality of generated clothing images still has room for improvement. Therefore, how to improve the quality of generated clothing images when using sketches and texture maps as input is still a technical problem that needs to be solved urgently in the prior art.
[0085] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a clothing design method based on bidirectional feature fusion of sketch and texture map, including step S101 to step S102, each step is as follows:
[0086] Step S101: performing multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed to obtain multiple fused feature pairs; wherein the sketch features and the texture map features are respectively generated based on the sketch and texture map of the clothing to be designed; in each round of feature fusion, adjusting the input sketch features based on the input texture map features to obtain output sketch features, and adjusting the input texture map features based on the input sketch features to obtain output texture map features, and taking the combination of the output sketch features and the output texture map features as the output fused feature pair.
[0087] In certain embodiments of the present application, the types of clothing to be designed include backpacks, tops, jackets, pants and skirts; the sketch is generated by the clothing to be designed through an edge detection algorithm; and the texture map is generated by the clothing to be designed through cropping and background masking.
[0088] Specifically, the preferred implementation of the edge detection algorithm is the Holistically-Nested Edge Detection algorithm; the preferred size of the cropping is 32×32; and the preferred implementation of the background mask is the Boundary-Aware Salient Object Detection algorithm.
[0089] It should be understood that the sketch features and the texture map features described in the present application are generated based on the sketch and texture map of the clothing to be designed, respectively, and the implementation methods thereof include but are not limited to: directly using the sketch and texture map of the clothing to be designed as the sketch features and texture map features; obtaining the sketch features and texture map features after feature conversion of the sketch and texture map of the clothing to be designed. The above implementation methods are well known to those skilled in the art, so they will not be described in detail here.
[0090] In certain embodiments of the present application, the step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes:
[0091] Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features;
[0092] Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair;
[0093] The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
[0094] See also Figure 3, is a bidirectional feature fusion network framework diagram shown in the implementation mode of this application, which is applied to multiple rounds of feature fusion; wherein, Figure 3 (a) is the specific operation process of multi-round feature fusion. Figure 3 (b) is the specific operation process of each round of feature fusion; S is the sketch, O is the texture map; s1, o1 are the sampled sketch features and sampled texture map features obtained after downsampling; Down Sample is the downsampling of the input features, G is the parameter extraction of the features, To pre-calculate the input features; is a sketch parameter pair, is a texture map parameter pair; f1, f2, f3, f4 are the output sketch features adjusted in each round of feature fusion, f′1, f′2, f′3 are the corresponding output texture map features; 1×1Conv is a 1×1 convolution operation, and ReLu is an activation operation; They are vector product operation and vector addition operation respectively.
[0095] Specifically, in the first round of feature fusion:
[0096] (1) Down-sample the input sketch feature S and texture map feature O to obtain the sampled sketch feature and sampled texture map features Among them, l, h, w, n are the number of feature map channels, image height, image width and resolution level respectively. Usually, l = 288, h = w = 256, n = 3.
[0097] (2) Parameter extraction G is performed on the sampled sketch feature s1 and the sampled texture map feature o1, namely and Get sketch parameter pair and texture map parameters
[0098] Specifically, in the feature fusion of the i-th round, except for the sampling sketch feature and sampled texture map features Except for this, the rest of the operations are the same as those in the first round of feature fusion and will not be described here.
[0099] It should be understood that in the application scenario of processing sketches and texture images for clothing design in the present application, the sketch parameter pair and the texture image parameter pair have actual physical meanings, wherein: represents the scaling factor, and Represents the translation factor.
[0100] In certain embodiments of the present application, the number of rounds of feature fusion is set to 4 according to the current task requirements, but it should be understood that in the scenario where the bidirectional feature fusion network framework of the present application is applied, based on the refinement of the tasks of the present application or the requirements of other tasks of the same category, the number of rounds of feature fusion is not limited to 4 and can be changed to other values.
[0101] The present application first downsamples the sketch features and the texture map features to obtain sampled sketch features and sampled texture map features, respectively, and then performs parameter extraction to obtain sketch parameter pairs and texture map parameter pairs, respectively, so as to obtain parameters that can adjust the output of each other, so that when the sampled sketch features are adjusted according to the texture map parameter pairs and the sampled texture map features are adjusted according to the sketch parameter pairs, the feature details of the sketch and the texture map are effectively fused, thereby improving the degree of mutual fusion between the sketch and the texture map.
[0102] In certain embodiments of the present application, the step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain the output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain the output texture map feature specifically includes:
[0103] The adjusting of the sampling sketch feature is specifically as follows:
[0104]
[0105] Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively;
[0106] The adjusting of the sampling texture map feature is specifically:
[0107]
[0108] Among them, f′ i To output texture features, is a sketch parameter pair, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
[0109] This application is based on feature fusion rounds, adjusts the sampled sketch features according to texture map parameters and adjusts the sampled texture map features according to sketch parameters, which can effectively fuse the feature details of the sketch and texture map and improve the degree of mutual fusion between the sketch and texture map.
[0110] Step S102: inputting the sketch, the texture map and the multiple fusion feature pairs into a first clothing generation model, generating based on a preset guiding scale, and obtaining a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
[0111] In certain embodiments of the present application, the first clothing generation model is obtained by training based on a preset clothing generation model, specifically including:
[0112] Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map;
[0113] The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
[0114] See also Figure 4 , is a structural diagram of a clothing design model framework of the present application, i.e., a structural diagram of a model of a first clothing generation model (preset clothing generation model); wherein S is a sketch, O is a texture map; bFMFus ion is a multi-round feature fusion, and Denoising Unet is a denoising network; Z T-1 is the predicted clothing design image obtained by the first denoising step, ×(T-1) represents repeated (T-1) steps of denoising operation; D represents the image decoding operation.
[0115] In certain embodiments of the present application, a preferred implementation of the denoising network Denoising Unet of the preset clothing generation model is Unet.
[0116] This application first obtains clothing image training data, and then inputs a preset clothing generation model for iterative training. During iteration, the model parameters are optimized through the overall generation loss composed of noise prediction loss, variational lower bound loss and style loss, which can enable the model to converge quickly to the task requirements and be more in line with the actual task of clothing generation.
[0117] In certain embodiments of the present application, each round of iteration optimizes the model parameters based on the overall generation loss, specifically including:
[0118] In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs;
[0119] Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise;
[0120] De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image;
[0121] Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph;
[0122] The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
[0123] In each round of iteration, the present application first adds noise to the clothing design drawing based on the preset noise to obtain a clothing noise map, and performs multiple rounds of feature fusion on the training sketch and the training texture map to obtain multiple training fusion feature pairs, so as to obtain basic training data, and then predicts the clothing noise map to obtain predicted noise, and calculates the noise prediction loss by combining the multiple training fusion feature pairs and the preset noise, so as to reasonably calculate the noise prediction loss based on the actual needs of the task; then, the clothing noise map is denoised based on the predicted noise to obtain a clothing prediction map, and then the style loss and the variational lower bound loss are calculated, so as to reasonably calculate the style loss and the variational lower bound loss based on the actual needs of the task; thus, when the noise prediction loss, the style loss and the variational lower bound loss are combined to obtain the overall generation loss and optimize the model parameters of the preset clothing generation model, it is more in line with the actual task of clothing generation.
[0124] In certain embodiments of the present application, before performing multiple rounds of feature fusion on the training sketch and the training texture map to obtain multiple training fusion feature pairs, the method further includes:
[0125] The training sketch and the training texture map are set and updated according to preset empty probabilities to obtain updated training sketch and training texture map.
[0126] In certain embodiments of the present application, the preferred value of the preset null probability is 0.1.
[0127] By pre-setting the empty probability to set and update the training sketches and training texture maps, compared with the existing training method, no additional classifier is required to guide the image sampling generation process, which can increase the learning scope of the model and improve the feature expression ability of the model.
[0128] In certain embodiments of the present application, the overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including:
[0129] L=Lsimple +L υlb +L style
[0130]
[0131] Among them, L, L simple , L υlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,..., T is the time step, T is the sequence length, z0=E(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
[0132] In some embodiments of the present application, the image encoding E(·) and the image decoding All are achieved based on a pre-trained autoencoder model. Specifically, the autoencoder model includes an encoder and a decoder, including but not limited to a variational autoencoder (VAE), a contractive autoencoder (CAE) and a regularized autoencoder (RAE), and a preferred implementation is a variational autoencoder VAE.
[0133] In certain embodiments of the present application, the VGG discriminant function The calculation is based on the Gram matrix of the mth layer of the pre-trained VGG19 model.
[0134] In certain embodiments of the present application, the step of inputting the sketch, the texture map, and the plurality of fusion feature pairs into a first clothing generation model, generating the clothing design drawing of the clothing to be designed based on a preset guidance scale, specifically includes:
[0135] Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing;
[0136] The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows:
[0137]
[0138] in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ (·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S, O are the sketch and texture map respectively, S4, O s To set the guideline scale;
[0139] The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.
[0140] The present application first inputs a sketch, a texture map and a plurality of fusion feature pairs into a first clothing generation model, and gradually performs denoising based on a preset guiding scale to generate a predicted clothing design drawing, which can stabilize the clothing generation scale, thereby improving the reliability of clothing design drawing generation when decoding the predicted clothing design drawing to obtain the clothing design drawing of the clothing to be designed.
[0141] Compared with the prior art, the present application first performs multiple rounds of feature fusion on the sketch and texture map of the clothing to be designed. In each round of feature fusion, the sketch and texture map are fused with each other to adjust each other's feature output, and multiple fused feature pairs are obtained. The feature details of the sketch and texture map can be effectively fused, which is beneficial to the migration of texture details on the sketch during subsequent clothing generation. The sketch, texture map and multiple fused feature pairs are then input into the first clothing generation model, and generation is performed based on a preset guidance scale. This can stabilize the clothing generation scale and improve the reliability of clothing design drawing generation.
[0142] Corresponding to the previous method, see Figure 2 , the embodiment of the present application provides a clothing design device based on bidirectional feature fusion of sketch and texture map, including a feature fusion module 210 and a clothing generation module 220;
[0143] The feature fusion module 210 is used to perform multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed, and obtain multiple fused feature pairs; wherein the sketch features and the texture map features are generated based on the sketch and texture map of the clothing to be designed, respectively; in each round of feature fusion, the input sketch features are adjusted based on the input texture map features to obtain output sketch features, and the input texture map features are adjusted based on the input sketch features to obtain output texture map features, and the combination of the output sketch features and the output texture map features is used as the output fused feature pair;
[0144] The clothing generation module 220 is used to input the sketch, the texture map and the multiple fusion feature pairs into a first clothing generation model, generate based on a preset guidance scale, and obtain a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
[0145] In certain embodiments of the present application, the step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes:
[0146] Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features;
[0147] Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair;
[0148] The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
[0149] In certain embodiments of the present application, the step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain the output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain the output texture map feature specifically includes:
[0150] The adjusting of the sampling sketch feature is specifically as follows:
[0151]
[0152] Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively;
[0153] The adjusting of the sampling texture map feature is specifically:
[0154]
[0155] Among them, f′ i To output texture features, is a pair of sketch parameters, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
[0156] In certain embodiments of the present application, the first clothing generation model is obtained by training based on a preset clothing generation model, specifically including:
[0157] Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map;
[0158] The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
[0159] In certain embodiments of the present application, each round of iteration optimizes the model parameters based on the overall generation loss, specifically including:
[0160] In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs;
[0161] Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise;
[0162] De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image;
[0163] Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph;
[0164] The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
[0165] In certain embodiments of the present application, the overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including:
[0166] L=L simple +L υlb +L style
[0167]
[0168] Among them, L, L simple , L υlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,..., T is the time step, T is the sequence length, z0=E(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
[0169] In certain embodiments of the present application, the step of inputting the sketch, the texture map, and the plurality of fusion feature pairs into a first clothing generation model, generating the clothing design drawing of the clothing to be designed based on a preset guidance scale, specifically includes:
[0170] Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing;
[0171] The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows:
[0172]
[0173] in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ(·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S and O are sketch and texture map respectively, S s , O s To set the guideline scale;
[0174] The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.
[0175] The present application first performs multiple rounds of feature fusion on the sketch and texture map of the clothing to be designed. In each round of feature fusion, the sketch and texture map are fused with each other to adjust each other's feature output, and multiple fused feature pairs are obtained. The feature details of the sketch and texture map can be effectively fused, which is beneficial to the migration of texture details on the sketch during subsequent clothing generation. The sketch, texture map and multiple fused feature pairs are then input into the first clothing generation model, and generation is performed based on a preset guidance scale. This can stabilize the clothing generation scale and improve the reliability of clothing design drawing generation.
[0176] It should be understood that the device provided in the embodiment of the present application corresponds to the aforementioned method. The clothing design device based on the bidirectional feature fusion of sketches and texture maps provided in the embodiment of the present application can realize the clothing design method based on the bidirectional feature fusion of sketches and texture maps provided in any embodiment of the present application.
[0177] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.
[0178] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;
[0179] Among them, when the processor executes the computer program, a clothing design method based on bidirectional feature fusion of sketch and texture map of the present application is implemented.
[0180] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute a clothing design method based on bidirectional feature fusion of sketches and texture images of the present application.
[0181] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A clothing design method based on bidirectional feature fusion of sketch and texture map, characterized in that: include: Performing multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed to obtain multiple fused feature pairs; wherein the sketch features and the texture map features are generated based on the sketch and texture map of the clothing to be designed, respectively; in each round of feature fusion, adjusting the input sketch features based on the input texture map features to obtain output sketch features, and adjusting the input texture map features based on the input sketch features to obtain output texture map features, and taking the combination of the output sketch features and the output texture map features as the output fused feature pair; The sketch, the texture map and the multiple fusion feature pairs are input into a first clothing generation model, and generated based on a preset guiding scale to obtain a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
2. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 1 is characterized in that: The step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes: Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features; Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair; The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
3. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 2 is characterized in that: The step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain an output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain an output texture map feature, specifically includes: The adjusting of the sampling sketch feature is specifically as follows: Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively; The adjusting of the sampling texture map feature is specifically: Among them, f′ i To output texture features, is a pair of sketch parameters, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
4. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 1 is characterized in that: The first clothing generation model is obtained by training based on a preset clothing generation model, and specifically includes: Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map; The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
5. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 4 is characterized in that: Each round of iteration optimizes the model parameters based on the overall generation loss, specifically including: In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs; Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise; De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image; Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph; The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
6. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 5 is characterized in that: The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including: Among them, L, L simple , L vlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,..., T is the time step, T is the sequence length, z0=E(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
7. The clothing design method based on bidirectional feature fusion of sketch and texture map according to claim 1 is characterized in that: The step of inputting the sketch, the texture map and the plurality of fusion feature pairs into a first clothing generation model and generating the clothing design drawing of the clothing to be designed based on a preset guiding scale specifically includes: Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing; The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows: in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ (·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S and O are sketch and texture map respectively, S s , O s To set the guideline scale; The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.
8. A clothing design device based on bidirectional feature fusion of sketch and texture map, characterized in that: Includes feature fusion module and clothing generation module; The feature fusion module is used to perform multiple rounds of feature fusion on the sketch features and texture map features of the clothing to be designed to obtain multiple fused feature pairs; wherein the sketch features and the texture map features are generated based on the sketch and texture map of the clothing to be designed respectively; in each round of feature fusion, the input sketch features are adjusted based on the input texture map features to obtain output sketch features, and the input texture map features are adjusted based on the input sketch features to obtain output texture map features, and the combination of the output sketch features and the output texture map features is used as the output fused feature pair; The clothing generation module is used to input the sketch, the texture map and the multiple fusion feature pairs into a first clothing generation model, generate based on a preset guidance scale, and obtain a clothing design drawing of the clothing to be designed; wherein the first clothing generation model is trained based on a preset clothing generation model.
9. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 8, characterized in that: The step of adjusting the input sketch feature based on the input texture map feature to obtain the output sketch feature, and adjusting the input texture map feature based on the input sketch feature to obtain the output texture map feature, specifically includes: Down-sampling the input sketch features and texture map features respectively to obtain sampled sketch features and sampled texture map features; Extracting parameters of the sampled sketch features and the sampled texture map features respectively to obtain a sketch parameter pair and a texture map parameter pair; The sampling sketch feature is adjusted according to the texture map parameter pair to obtain an output sketch feature, and the sampling texture map feature is adjusted according to the sketch parameter pair to obtain an output texture map feature.
10. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 9, characterized in that: The step of adjusting the sampled sketch feature according to the texture map parameter pair to obtain an output sketch feature, and adjusting the sampled texture map feature according to the sketch parameter pair to obtain an output texture map feature, specifically includes: The adjusting of the sampling sketch feature is specifically as follows: Among them, i is the feature fusion round, f i To export sketch features, is a texture map parameter pair, are the mean and variance of the sampled sketch features and the sampled sketch features, respectively; The adjusting of the sampling texture map feature is specifically: Among them, f′ i To output texture features, is a pair of sketch parameters, are the mean and variance of the sampled texture map features and the sampled texture map features, respectively.
11. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 8, characterized in that: The first clothing generation model is obtained by training based on a preset clothing generation model, and specifically includes: Acquire clothing image training data; wherein the clothing image training data is composed of multiple clothing image groups; wherein each clothing image group includes a clothing design drawing and a corresponding training sketch and training texture map; The clothing image training data is input into a preset clothing generation model for iterative training. In each round of iteration, the model parameters are optimized based on the overall generation loss until the model parameters meet a preset convergence threshold, and the training is completed to obtain a first clothing generation model; wherein the overall generation loss includes noise prediction loss, variational lower bound loss and style loss.
12. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 11, characterized in that: Each round of iteration optimizes the model parameters based on the overall generation loss, specifically including: In each round of iteration, noise is added to the clothing design drawing based on preset noise to obtain a clothing noise map, and multiple rounds of feature fusion are performed on the training sketch and the training texture map to obtain multiple training fusion feature pairs; Predicting the clothing noise image to obtain predicted noise, and calculating the noise prediction loss according to the multiple training fusion feature pairs, the preset noise and the predicted noise; De-noising the clothing noise image based on the predicted noise to obtain a clothing prediction image; Calculating the style loss and the variational lower bound loss according to the clothing design graph and the clothing prediction graph; The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, and the model parameters of the preset clothing generation model are optimized according to the overall generation loss.
13. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 12, characterized in that: The overall generation loss is obtained according to the noise prediction loss, the style loss and the variational lower bound loss, specifically including: Among them, L, L simple , L vlb , L style They are the overall generation loss, noise prediction loss, variational lower bound loss and style loss, ∈ t is the preset noise, t=1,..., T is the time step, T is the sequence length, z0=F(x0), z t are clothing design images and clothing noise images, respectively. E(·) represents image encoding, x0 is the clothing design image, To predict noise, F(·,··) represents multiple rounds of feature fusion, S and O are training sketches and training texture maps respectively, ||·|| is the 2-norm, is the prediction loss calculation function, is the VGG discriminant function, Predict images for clothing, Indicates image decoding, For clothing prediction chart, is the accumulated noise intensity.
14. The clothing design device based on bidirectional feature fusion of sketch and texture map according to claim 8, characterized in that: The step of inputting the sketch, the texture map and the plurality of fusion feature pairs into a first clothing generation model and generating the clothing design drawing of the clothing to be designed based on a preset guiding scale specifically includes: Inputting the sketch, the texture map and the plurality of fused feature pairs into a first clothing generation model, and gradually removing noise based on a preset guiding scale to obtain a predicted clothing design drawing; The stepwise denoising based on the preset guiding scale to obtain the predicted clothing design drawing is specifically as follows: in, The predicted clothing design image generated by denoising the (t-1)th time node, z t is the predicted clothing design image generated by denoising the previous time node of the (t-1)th time node, α t is the noise intensity, β t is the noise variance, is the cumulative noise intensity, To correct the noise, ∈ θ (·, ··) is the predicted noise, F(·, ··) represents multi-round feature fusion, S and O are sketch and texture map respectively, S s , O s To set the guideline scale; The predicted clothing design drawing is decoded to obtain a clothing design drawing of the clothing to be designed.