A clothing image generation method for disentangled representation of design attributes

Abstract

The invention discloses a clothing image generation method for design attribute disentanglement representation, which utilizes a global design attribute coupling network to disentangle geometric shape, texture and shadow design attributes from clothing images in a store, and couples to generate clothing images with global design attributes; and utilizes a local design attribute coupling network to disentangle decorative design attributes from a source clothing image with decorative design attributes, and fuses the decorative design attributes with the clothing image with global design attributes output by the global design attribute coupling network to obtain a final coupled image; the invention is suitable for automatic generation of clothing design images, and the method helps to solve the problem of slow response in the process from demand acquisition to design feedback, helps to visualize real-time offline clothing, and helps to simplify complicated clothing design process.

Description

Technical Field

[0001] The present invention belongs to the field of cross-domain conversion of clothing images, and in particular relates to a clothing image generation method oriented to disentangled representation of design attributes. Background Art

[0002] When shopping for clothing online, consumers are often dissatisfied with the existing clothing styles. However, personalized clothing customization that meets users' aesthetic needs is expensive and difficult to promote effectively. In clothing design, designers need to go through a complicated process to transform design concepts into material representations. Repetitive manual labor limits the creation of clothing designs. Personalized clothing customization and clothing design are arts about the visual characteristics of clothing images. The technology of quickly transforming user needs or design concepts into visible clothing images is particularly critical.

[0003] Theoretical research on cross-domain conversion mainly represents the structured disentanglement of generalized objects at the perceptual level. It has outstanding performance in domain conversion with large spans, but is not applicable to the clothing field with fine subdomain division. For existing clothing image translation technology, the attribute manipulation of the reference image is a multimodal problem. The generated clothing images can meet the requirements of attribute manipulation but may not meet the needs of shoppers and designers. It can only be applied to clothing recommendations, not to clothing customization or clothing design. Due to the lack of clustering of clothing image data and the generation of clothing subdomains, clothing cross-domain conversion can only be guided by text rather than visual form. Text guidance is not intuitive and can only deal with specific attributes. In addition, the images generated by clothing image translation are blurry and lack details, which affects its promotion in clothing customization and clothing design.

[0004] In summary, it is difficult for existing technologies to achieve image generation of clothing attribute subdomains based on visual features and guided by examples.

[0005] The purpose of the present invention is to provide a method for generating clothing images based on disentangled representation of design attributes in view of the deficiencies in the prior art. The present invention is aimed at the demand response in personalized clothing design and the generation from concept to image in clothing design. By disentangling the design attributes of the reference image and coupling the design attributes using a generative adversarial network, a clothing image with a new style is generated.

[0006] The objective of the present invention is achieved through the following technical solutions: the clothing domain is divided into geometry, texture, shadow and decoration subdomains according to design attributes. The design attributes are disentangled from the input using geometry masks, repeated texture region annotations, Laplacian image gradients and local landmarks or patterns. The design attributes are further clustered into global and local design attributes, and the design attributes are coupled using a global design attribute coupling network and a local design attribute coupling network with a parallel encoder-bottleneck-decoder structure, as well as a specific coupling operation to generate clothing images with new styles. In self-supervised and unsupervised learning, a multi-objective function is used in combination to constrain the coupling of design attributes in the generated images. Summary of the invention

[0007] The purpose of the present invention is to address the deficiencies of the prior art and provide a method for generating clothing images oriented to the disentanglement representation of design attributes.

[0008] The purpose of the present invention is specifically achieved through the following technical solutions: A method for generating clothing images for design attribute disentanglement representation, comprising the following steps:

[0009] Step (1): Use the global design attribute coupling network to obtain clothing images x from the store real Disentangle geometry, texture and shading design attributes and couple them to generate clothing images with global design attributes The global design attributes of clothing images in stores are disentangled using semantic segmentation, repeated region labeling, and Laplacian operators. A generator consisting of an encoder-bottleneck layer-decoder structure is used to learn the global design attributes, and a mixture of multiplication and adders is used to couple the global design attributes again to output clothing images with global design attributes. And design objective function for global design attribute coupling network;

[0010] Step (2): Utilize the local design attribute coupling network to extract the clothing image x with decorative design attributes from the source deco The decorative design attributes are disentangled and coupled with the global design attributes to output the clothing image with global design attributes. Fusion generates the final coupled image A fine-grained semantic segmentation network is used to extract the decorative or logo features of clothing images with decorative design attributes. Affine transformation is used to achieve strong positioning of decorative design attributes. The feature map of design attributes is learned through the encoder-bottleneck layer-decoder structure. The linear operation mask enables the network to have different attention, realize the natural transition between decorative design attributes and global design attributes, and output the final coupled image. And the objective function of the network design is coupled with local design attributes.

[0011] Furthermore, the step (1) is specifically as follows:

[0012] (1.1) The clothing image x in the store real Input the first decoupling module in the global design attribute coupling network for disentanglement, the clothing image x in the store real can be regarded as clothing images x with geometric shape design attributes. shap , clothing image x with texture design attributes text or clothing image x with shadow design attributes shad , the clothing image x with geometric shape design attributes shap Clothing image mask obtained after unwrapping The clothing image x with texture design attributes text The texture image formed by the repetitive area of ​​the array is obtained by unwrapping The clothing image x with shadow design attributes shad After unwrapping, the Laplace image gradient is obtained

[0013] (1.2) Mask the clothing image obtained in step (1.1) Texture image formed by the repeated areas of the array and the Laplacian image gradient Input the generator in the first coupling module in the global design attribute coupling network, and finally obtain the clothing image with global design attributes The generator is an encoder-bottleneck layer-decoder structure;

[0014] (1.3) The clothing image with global design attributes obtained in step (1.2) The texture attribute discriminator D is input to the first coupling module in the global design attribute coupling network. text and shadow attribute discriminator D shad In the above equation, we get the first result matrix ψ text and the second result matrix ψ shad ;

[0015] (1.4) Designing the objective function for the global design attribute coupling network includes designing an adversarial loss function for the discriminator in the global design attribute coupling network and designing an overall objective function for the generator in the global design attribute coupling network.

[0016] Furthermore, the step (1.1) is specifically as follows:

[0017] (1.1.1) For clothing images x with geometric design attributes shap Generating Clothing Image Masks Using a Semantic Segmentation Network Masking in clothing images In the example, binary (1 or 0) is used to distinguish clothing objects from background; the clothing image mask Only geometric shape design attributes are extracted from clothing images, without texture and shadow design attributes;

[0018] (1.1.2) Extracting clothing images x with texture design attributes by marking landmarks of repetitive texture regions text The texture design attribute of the array is to fill the texture image formed by the repeated texture area of ​​the array The size of x text Same; the texture image formed by the array repetitive area Only the texture design attributes are represented from the clothing image, and the geometric shape and shadow design attributes of the clothing image are not included;

[0019] (1.1.3) The Laplacian operator is used to extract the clothing image x with shadow design attributes shad The shadow design attribute of the clothing image x with shadow design attribute is unwrapped shad The brightness gradient feature is obtained by first Laplacian image gradient The first Laplacian image gradient It only represents the brightness gradient features in the shadow design attributes of clothing images, and discards the geometric shape and texture design attributes of clothing images;

[0020] The first Laplacian image gradient Using formula (1), we can calculate:

[0021]

[0022] Where g(·) is the color to grayscale image conversion function; is the convolution operator; k l is the Laplace convolution kernel, whose size is 3×3, and generates the second-order differential of the clothing image by calculating the grayscale gradient of the eight neighborhoods. Set the Laplacian convolution kernel k l The step size is 1, and the Laplacian convolution operation filling is set to 1; the first Laplacian image gradient The size of x shad are the same size;

[0023] (1.1.4) Mask the clothing image generated in step (1.1.1) The texture image formed by the array repetitive area obtained in step (1.1.2) First, perform element-wise multiplication to obtain a clothing image x with geometric shape and texture design attributes. fake ,Right now In the formula, ⊙ is the symbol of element product; then fake Use the Laplacian operator to extract the shadow design attributes and obtain the second Laplacian image gradient Right now

[0024] (1.1.5) The first Laplacian image gradient obtained in step (1.1.3) and the second Laplacian image gradient Merge into Laplacian image gradient

[0025] Furthermore, the step (1.2) is specifically as follows:

[0026] (1.2.1) Mask the clothing image Input geometry encoder E shap Get the geometric shape feature map Texture image formed by the repeated areas of the array Input texture encoder E text Get texture feature map Laplacian Image Gradient Input shadow encoder E shad Get the shadow feature map Said Where H1, W1 and C1 are the feature map height, width and number of channels, and the value of the channel number C1 is 128; the geometric shape encoder E shap , Texture Encoder E text and Shadow Encoder E shad have the same structure;

[0027] (1.2.2) The geometric shape feature map Texture feature map and shadow feature map Parallel input bottleneck layer 1, the bottleneck layer 1 includes 3×3 bottleneck layers, each of which is composed of two {two-dimensional convolutional layers, instance normalization layers, activation layers} combined in series; geometric shape feature map Output after 3 bottleneck layers Texture feature map β1 0 After 3 bottleneck layers, output Shadow feature map After 3 bottleneck layers, output In the gap between bottleneck layers, geometry, texture, and shading design properties are coupled again: in Where ⊙ is the element product;

[0028] (1.2.3) Masking clothing images Downsampling to get the downsampling function The coupling result of step (1.2) Input bottleneck layer 2, which is composed of 3 bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; coupling result Input the first bottleneck layer of bottleneck layer 2 to get and Coupling Then will Input the second bottleneck layer of bottleneck layer 2 to get Again with Coupling Then Input the third bottleneck layer of bottleneck layer 2 to get Again with Coupling

[0029] (1.2.4) will eventually Input to the decoder outputs a clothing image with global design attributes Said Masking from clothing images Texture image formed by the repeated areas of the array and the Laplacian image gradient The geometry, texture, and shading design properties are coupled.

[0030] Furthermore, the step (1.3) is specifically as follows:

[0031] (1.3.1) The clothing image with global design attributes obtained in step (1.3) Cropping to texture design attribute representation image Said The cropping area is as follows:

[0032] {(x,y)∈R|x∈[Ο(x)-W2 / 2,Ο(x)+W2 / 2],y∈[Ο(y)-H2 / 2,Ο(y)+H2 / 2]}

[0033] Where (x, y) is the texture design attribute representation image Pixel coordinates in the image; Ο(x) and Ο(y) are texture design attribute representation images The x and y coordinate values ​​of the center point; H2 and W2 are texture design attribute representation images The height and width of

[0034] The texture design attributes are then used to represent the image Input to the texture attribute discriminator D text Generate the first result matrix ψ text , Where H3 and W3 are the first result matrix ψ text The height and width of the texture attribute discriminator D text It consists of four combined blocks {2D convolutional layer, batch normalization layer, leaky activation layer};

[0035] (1.3.2) Decoupling clothing images with global design attributes using the Laplacian operator The shadow design attribute of the image is generated Then the shadow design attribute represents the image Input shadow attribute discriminator D shad , and get the second result matrix ψ shad , Where H4 and W4 are the second result matrix ψ shad The height and width of the

[0036] Furthermore, the step (1.4) is specifically as follows:

[0037] (1.4.1) Global Design Attributes Adversarial Loss Function for Discriminators in Coupled Networks: Texture Attribute Discriminator D text and shadow attribute discriminator D shad The adversarial loss function The definition is as follows:

[0038]

[0039] Where log[·] is the logarithmic function; Design attribute representation images for textures; Design attribute representation images for shadows; is the first Laplacian image gradient; is the second Laplace image gradient; E{·} is the expected value; x′ text To crop the label image The acquired image;

[0040] Said Used to constrain the texture attribute discriminator D in the global design attribute coupling network text and shadow attribute discriminator D shad Optimized reverse;

[0041] (1.4.2) The overall objective function design of the generator in the global design attribute coupling network:

[0042] (1.4.2.1) Generator adversarial loss function: Design adversarial loss function for the generator in the coupled network with global design attributes Said is defined as follows:

[0043]

[0044] In the formula, is the weight coefficient;

[0045] (1.4.2.2) Global L1 loss objective function: transform the clothing image x in the store real and an image x with geometric shape and texture design attributes fake As label images, clothing images with global design attributes As a result, a global L1 loss objective function is constructed Said is defined as follows:

[0046]

[0047] In the formula, is the weight coefficient;

[0048] (1.4.2.3) VGG loss objective function: Use the pre-trained VGG-16 model to constrain the global design coupling network at the perception level, taking the clothing image x in the store real and an image x with geometric shape and texture design attributes fake As label images, clothing images with global design attributes Construct the VGG loss objective function as the generated result Said is defined as follows:

[0049]

[0050] In the formula, vgg i (·) is the i-th layer output of the VGG-16 model, with a total of 4 layers of output; and is the label image weight; x real and x fake is the label image;

[0051] (1.4.2.4) Coupling consistency loss objective function: Assuming that the clothing image has global design attributes The unwrapped geometry and texture design properties remain x shap and x text , and the untangled shadow design properties are solved by the Laplacian operator; therefore, the coupled consistency loss objective function is defined as follows:

[0052]

[0053] In the formula, and is the weight coefficient of the label image, G GDA (·) is the output result of the generator in the global design attribute coupling network;

[0054] (1.4.2.5) Gradient loss objective function: In order to further strengthen the coupling of shadow design attributes, the shadow design attribute representation image is constrained at full size The first Laplacian image gradient and the second Laplacian image gradient As the label image and compared with it, we get the gradient loss objective function Said is defined as follows:

[0055]

[0056] In the formula, and is the weight coefficient of the label image;

[0057] (1.4.2.6) The total objective function of the generator in the global design attribute coupling network: The total objective function of the generator in the global design attribute coupling network is defined as follows:

[0058]

[0059] Where λ1, λ2, λ3, λ4 and λ5 are the hyperparameters of each loss function;

[0060] Said Optimized inverse of coupled network generators used to constrain global design properties.

[0061] Furthermore, the step (2) is specifically as follows:

[0062] (2.1) The source clothing image x has decorative design attributes deco Input the second decoupling module in the local design attribute coupling network for disentanglement to obtain a clothing image with decorative design attributes

[0063] For the input source clothing image x with decorative design attributes deco Perform affine transformation to obtain T·x deco , and then use a fine-grained semantic segmentation network or manual segmentation to obtain the segmentation map x′ deco, and T·x deco With the segmentation map x′ deco Perform element-wise multiplication to obtain clothing images with decorative design attributes Right now Where T∈R 3 ×3 is an affine matrix, a1~a4 are rotation and scaling coefficients; t1 and t2 are translation coefficients;

[0064] (2.2) Clothing images with decorative design attributes Garment images with global design attributes Clothing Image Mask and the partition map x′ deco The generator in the second coupling module in the local design attribute coupling network is input to obtain the final coupling image The generator is an encoder-bottleneck layer-decoder structure;

[0065] (2.3) The design objective function of the local design attribute coupling network is the overall objective function of the generator design in the local design attribute coupling network.

[0066] Furthermore, the step (2.2) is specifically as follows:

[0067] (2.2.1) Using the segmentation graph x′ deco and clothing image mask Strengthen the attention of the local design attribute coupling network to the overall contour and local decoration of the clothing; define the attention mask x″ of the non-decoration part in the clothing object deco as follows:

[0068]

[0069] Multiply This is to prevent the image from belonging to x′ deco But not The part causes threshold leakage;

[0070] (2.2.2) Clothing images with global design attributes Input encoder E glob Get the feature map η0, and transform the clothing image with decorative design attributes Input encoder E deco Get the feature map μ0, Among them, H1, W1 and C1 are the height, width and number of channels of the feature map, and the value of the number of channels C1 is 128;

[0071] (2.2.3) Attention mask x″ for the non-decorative parts of the clothing object decoDownsampling obtains the downsampling function γ(x′ deco ); for the segmentation graph x′ deco Downsampling obtains the downsampling function γ(x′ deco );Mask the clothing image Downsampling to get the downsampling function

[0072] (2.2.4) The feature map η0, feature map μ0, downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Input bottleneck layer 3, the bottleneck layer includes 2×3 bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; the feature map η0 is obtained by passing through the bottleneck layer to obtain η1, and the feature map μ0 is obtained by passing through the bottleneck layer to obtain μ1, and the downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Coupling is performed to obtain η1′: Where ⊙ is the element product symbol, b1 and b2 are the superposition coefficient and substitution coefficient respectively; η′1 is coupled again to obtain η′2: η′2 is coupled again to obtain η′3:

[0073] (2.2.5) Finally, η′3 is input into the decoder to output the final coupled image The final coupled image It realizes the extraction of clothing images with decorative design attributes. Garment images with global design attributes Clothing Image Mask and the partition map x′ deco Get geometry, texture, shading and decorative design properties from

[0074] Furthermore, the step (2.3) is specifically as follows:

[0075] (2.3.1) First, design the local L1 loss objective function; transform the clothing image with global design attributes and clothing images with decorative design attributes As the label image, the final coupled image As the generated result, a local L1 loss objective function is constructed The local L1 loss objective function is defined as follows:

[0076]

[0077] In the formula, λglob ,λ loca is the corresponding weight coefficient;

[0078] (2.3.2) Secondly, we design the local L2 loss objective function: In order to make the clothing image with global design attributes and clothing images with decorative design attributes The transition between them is natural, using the local L2 loss objective function To blur the boundary, the local L2 loss objective function The definition of

[0079]

[0080] In the formula, λ glob ,λ loca is the corresponding weight coefficient;

[0081] (2.3.3) Finally, the local L1 loss objective function and the local L2 loss objective function are used to design the total objective function of the generator in the local design attribute coupling network: the total objective function L of the generator in the local design attribute coupling network LDA is defined as follows:

[0082]

[0083] Where λ6 and λ7 are the hyperparameters of the loss function;

[0084] The L LDA Optimization inverse of coupled network generators used to constrain local design properties.

[0085] The beneficial effects of the present invention are as follows: the present invention is mainly based on a clothing generation method for design attribute disentanglement representation, which can help solve the problem of slow response from demand acquisition to feedback in clothing customization, and realize offline real-time clothing demand visualization feedback. The method can help simplify the complicated process of clothing design and realize the automatic generation from design concept to clothing image. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a structural diagram of the global design attribute coupling network;

[0087] Figure 2 It is a structural diagram of the local design attribute coupling network;

[0088] Figure 3 Structural diagram of the generator for the global design attribute coupling network;

[0089] Figure 4 Structural diagram of the generator for locally designed attribute coupling networks;

[0090] Figure 5 is the clothing picture of test set 1 in Example 2;

[0091] Figure 6 is the clothing picture of test set 2 in Example 2;

[0092] Figure 7 This is the clothing picture of test set 3 in Example 2. DETAILED DESCRIPTION

[0093] In order to explain the present invention more clearly, the present invention is further described below in conjunction with the accompanying drawings and embodiments. Those skilled in the art should understand that the content described below is illustrative rather than restrictive and should not limit the scope of protection of the present invention.

[0094] At the visual level, the "style" of clothing is disentangled into geometric shapes, textures, shadows, and decorative design attributes. All design attributes are further clustered into global design attributes and local design attributes. Global design attributes characterize the overall characteristics of clothing images, including geometric shapes, textures, and shadow features, while local design attributes characterize the detailed characteristics of clothing images, including decorative design attributes. Clustering design attributes can eliminate the representation interference of global and local design attributes, prevent the generation of noise, and generate clearer coupled decoration or logo features in the generated clothing images. The global design attribute coupling network is trained using a self-supervised learning method, and the local design attribute coupling network is trained using an unsupervised learning method to generate clothing images with specific design attributes for clothing customization and clothing design.

[0095] Example 1

[0096] like Figure 1-Figure 2 As shown, the present invention proposes a clothing image generation method for design attribute disentanglement representation, comprising the following steps:

[0097] Step (1): Use the global design attribute coupling network to obtain clothing images x from the store real Disentangle geometry, texture and shading design attributes and couple them to generate clothing images with global design attributes The global design attributes of clothing images in stores are disentangled using semantic segmentation, repeated region labeling, and Laplacian operators. A generator consisting of an encoder-bottleneck layer-decoder structure is used to learn the global design attributes, and a mixture of multiplication and adders is used to couple the global design attributes again to output clothing images with global design attributes. And design objective function for global design attribute coupling network;

[0098] Step (1) is implemented through the following sub-steps:

[0099] (1.1) The clothing image x in the storereal Input the first decoupling module in the global design attribute coupling network for disentanglement, the clothing image x in the store real can be regarded as clothing images x with geometric shape design attributes. shap , clothing image x with texture design attributes text or clothing image x with shadow design attributes shad , the clothing image x with geometric shape design attributes shap Clothing image mask obtained after unwrapping The clothing image x with texture design attributes text The texture image formed by the repetitive area of ​​the array is obtained by unwrapping The clothing image x with shadow design attributes shad After unwrapping, the Laplace image gradient is obtained

[0100] The clothing image x in the store real The following conditions should be met: a)x real It should be an image of clothing hanging on a hanger in a store or worn on a mannequin. After initial alignment, the clothing object should be centered in the image and have a high pixel ratio; b) x real It should have clear outlines, no structural interference, repetitive texture features and obvious shadows or regional brightness gradients; c) x real All global design properties coupled in the design should not be disturbed by local patterns or logos.

[0101] (1.1.1) For a clothing image x with geometric design attributes shap , it is necessary to disentangle the geometric shape design attributes while discarding the texture and shadow design attributes; therefore, for the clothing image x with geometric shape design attributes shap Generating Clothing Image Masks Using a Semantic Segmentation Network Masking in clothing images In the example, binary (1 or 0) is used to distinguish clothing objects from background; the clothing image mask Only the geometric shape design attributes are extracted from the clothing images, without the texture and shadow design attributes.

[0102] (1.1.2) For a clothing image x with texture design attributes text , the texture properties of clothing should characterize the clothing material in a flat state without being affected by lighting conditions. Therefore, the texture design properties of clothing images with texture design properties are extracted by marking landmarks of repetitive texture regions, and the texture image formed by filling the array repetitive texture regions is formed by array repetitive texture regions. The size of x textThe same; thus, the texture image formed by the array repetitive area Only the texture design attributes are represented from the clothing image, and the geometry and shading design attributes of the clothing image are not included.

[0103] (1.1.3) For a clothing image x with shadow design attributes shad , using the clothing image gradient to indirectly characterize the shadow design attributes of the clothing image; using the Laplacian operator to extract the clothing image x with shadow design attributes shad The gradient of the clothing image is obtained by using the Laplacian operator to characterize the shadow design attributes of the clothing image, while the geometric shape and texture design attributes of the clothing image are discarded.

[0104] The clothing image gradient consists of two parts: texture gradient and brightness gradient; the texture gradient is caused by the color change between clothing textures, and the brightness gradient is formed by the reflection of ambient light; the texture gradient and the brightness gradient are deeply coupled, so that the brightness gradient cannot be independently derived from the clothing image x with shadow design attributes. shad On the other hand, if the original texture gradient is not coupled in the generated clothing image, the generated clothing image is actually subjected to median filtering, resulting in blurred texture details of the generated image. Therefore, the Laplacian operator is used to extract the clothing image x with shadow design attributes. shad The shadow design attribute of the clothing image x with shadow design attribute is unwrapped shad The brightness gradient feature is obtained by first Laplacian image gradient The first Laplacian image gradient Only the brightness gradient features in the shadow design attributes of the clothing image are characterized, and the geometric shape and texture design attributes of the clothing image are discarded.

[0105] The first Laplacian image gradient Using formula (1), we can calculate:

[0106]

[0107] Where g(·) is the color to grayscale image conversion function; is the convolution operator; k l is the Laplace convolution kernel, whose size is 3×3, and generates the second-order differential of the clothing image by calculating the grayscale gradient of the eight neighborhoods. Set the Laplacian convolution kernel k l The step size is 1, and the Laplacian convolution operation filling is set to 1; the first Laplacian image gradient The size of x shad are the same size.

[0108] (1.1.4) Mask the clothing image generated in step (1.1.1) The texture image formed by the array repetitive area obtained in step (1.1.2) First, perform element-wise multiplication to obtain a clothing image x with geometric shape and texture design attributes. fake ,Right now In the formula, ⊙ is the symbol of element product; then fake Use the Laplacian operator to extract the shadow design attributes and obtain the second Laplacian image gradient Right now Because x fake Only the geometric shape and texture design attributes of the clothing image are represented, so the second Laplacian image gradient Only the texture gradient features in the shadow design attributes of the clothing image are characterized, and the geometric shape and texture design attributes of the clothing image are discarded.

[0109] (1.1.5) The first Laplacian image gradient obtained in step (1.1.3) and the second Laplacian image gradient Merge into Laplacian image gradient

[0110] (1.2) Figure 3 As shown, the clothing image obtained in step (1.1) is masked Texture image formed by the repeated areas of the array and the Laplacian image gradient Input the generator in the first coupling module in the global design attribute coupling network, and finally obtain the clothing image with global design attributes The generator is an encoder-bottleneck layer-decoder structure;

[0111] (1.2.1) Mask the clothing image Input geometry encoder E shap Get the geometric shape feature map Texture image formed by the repeated areas of the array Input texture encoder E text Get texture feature map Laplacian Image Gradient Input shadow encoder E shad Get the shadow feature map Said Where H1, W1 and C1 are the feature map height, width and number of channels, and the value of the channel number C1 is 128; the geometric shape encoder E shap , Texture Encoder E text and Shadow Encoder E shadThey have the same structure but do not share parameters with each other, from which they learn the geometric shape, texture and shadow design properties of clothing, while training affine parameters to improve the robustness to non-aligned clothing images;

[0112] (1.2.2) The geometric shape feature map Texture feature map and shadow feature map Parallel input bottleneck layer 1, the bottleneck layer 1 includes 3×3 bottleneck layers, each of which is composed of two {two-dimensional convolutional layers, instance normalization layers, activation layers} combined in series; geometric shape feature map After 3 bottleneck layers, output Texture feature map After 3 bottleneck layers, output Shadow feature map After 3 bottleneck layers, output In the gap between bottleneck layers, geometry, texture, and shading design properties are coupled again: in Where ⊙ is the element product;

[0113] (1.2.3) Masking clothing images Downsampling to get the downsampling function The coupling result of step (1.2) Input bottleneck layer 2, which is composed of 3 bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; coupling result Input the first bottleneck layer of bottleneck layer 2 to get and Coupling Then will Input the second bottleneck layer of bottleneck layer 2 to get Again with Coupling Then Input the third bottleneck layer of bottleneck layer 2 to get Again with Coupling Compared with bottleneck layer 1, the output of bottleneck layer 2 is and It has strong attention, where the non-clothing object parts in the image are rigidly constrained to be zero, causing gradient vanishing in the error feedback calculation, ensuring that the geometric shape design properties are coupled in the generated results.

[0114] (1.2.4) will eventually Input to the decoder outputs a clothing image with global design attributes The clothing image with global design attributes Masking from clothing images Texture image formed by the repeated areas of the array and the Laplacian image gradient The geometry, texture, and shading design properties are coupled.

[0115] (1.3) The clothing image with global design attributes obtained in step (1.2) The texture attribute discriminator D is input to the first coupling module in the global design attribute coupling network. text and shadow attribute discriminator D shad middle;

[0116] (1.3.1) In order to focus on the texture design attributes, the clothing image with global design attributes obtained in step (1.3) is first Cropping to texture design attribute representation image Said The cropping area is as follows:

[0117] {(x,y)∈R|x∈[Ο(x)-W2 / 2,Ο(x)+W2 / 2],y∈[Ο(y)-H2 / 2,Ο(y)+H2 / 2]}

[0118] Where (x, y) is the texture design attribute representation image Pixel coordinates in the image; Ο(x) and Ο(y) are texture design attribute representation images The x and y coordinate values ​​of the center point; H2 and W2 are texture design attribute representation images The height and width of

[0119] The texture design attributes are then used to represent the image Input to the texture attribute discriminator D text Generate the first result matrix ψ text , Where H3 and W3 are the first result matrix ψ text The height and width of the texture attribute discriminator D text It consists of four combined blocks {2D convolutional layer, batch normalization layer, leaky activation layer}.

[0120] (1.3.2) Decoupling clothing images with global design attributes using the Laplacian operator The shadow design attribute of the image is generated Then the shadow design attribute represents the image Input shadow attribute discriminator D shad , and get the second result matrix ψshad , Where H4 and W4 are the second result matrix ψ shad The height and width of the

[0121] (1.4) Designing an objective function for the global design attribute coupling network includes designing an adversarial loss function for the discriminator in the global design attribute coupling network and designing an overall objective function for the generator in the global design attribute coupling network;

[0122] (1.4.1) Adversarial loss objective function; Texture attribute discriminator D is used in the global design attribute coupling network text and shadow attribute discriminator D shad , texture attribute discriminator D text and shadow attribute discriminator D shad The adversarial loss functions are

[0123]

[0124] Where log[·] is the logarithmic function; Design attribute representation images for textures; Design attribute representation images for shadows; is the first Laplacian image gradient; is the second Laplace image gradient; E{·} is the expected value; x t ' ext To crop the label image The acquired image;

[0125] Said Used to constrain the texture attribute discriminator D in the global design attribute coupling network text and shadow attribute discriminator D shad Optimized reverse; (1.4.2.1)

[0127] The adversarial loss function of the generator of the global design attribute coupled network is

[0128]

[0129] In the formula, is the weight coefficient.

[0130] (1.4.2.2) L1 loss objective function: L1 loss represents the pixel error between the generated result and the label image. It is more robust to noise and can reduce the blurring of the generated result.

[0131] x the clothing images in the store realand an image x with geometric shape and texture design attributes fake As label images, clothing images with global design attributes Construct L1 loss objective function as the generated result Said is defined as follows:

[0132]

[0133] In the formula, is the weight coefficient;

[0134] L1 loss objective function The purpose is to prevent the transfer function of the global design attribute coupling network from degenerating into The conversion function is If only x real As the label image, it is easy to simultaneously shad Geometry, texture and shading design attributes are coupled to the generated image That is, infer the source image from the clothing gradient. By attaching the label image x fake ,constraining each branch of the global design property coupling network to ,complete their respective coupling tasks;

[0135] (1.4.2.3) VGG loss objective function; use the pre-trained VGG-16 model to constrain the global design coupling network at the perception level; different from L1 loss, VGG loss focuses on the semantic features of the generated image, which is also in line with the coupling intention of the design attribute. VGG loss definition as follows:

[0136]

[0137] In the formula, vgg i (·) is the i-th layer output of the VGG-16 model, with a total of 4 layers of output; and is the label image weight; x real and x fake is the label image;

[0138] (1.4.2.4) Coupling consistency loss objective function; The final clothing image with global design attributes generated in the generator in the first coupling module Coupling all design attributes, including geometry, texture, and shading attributes; to ensure the consistency of this coupling, it is hoped that all design attributes can be obtained from the clothing image with global design attributes Untangle again and recouple to generate clothing images with global design attributes Shadow design attributes are the key to reflecting the authenticity of generated images, so it is assumed that clothing images with global design attributes The unwrapped geometry and texture design properties remain x shap and x text , and the untangled shadow design properties are solved by the Laplacian operator; therefore, the coupled consistency loss objective function is defined as follows:

[0139]

[0140] In the formula, and is the weight coefficient of the label image, G GDA (·) is the output result of the generator in the global design attribute coupling network

[0141] (1.4.5) Gradient loss objective function; In order to further strengthen the coupling of shadow design attributes, the shadow design attribute representation image is constrained at full size The first Laplacian image gradient and the second Laplacian image gradient As the label image and compared with it, we get the gradient loss objective function

[0142]

[0143] In the formula, and is the weight coefficient of the label image.

[0144] (1.4.6) Overall objective function. The overall objective function of the global design attribute coupling network is as follows:

[0145]

[0146] Where λ1, λ2, λ3, λ4 and λ5 are the hyperparameters of each loss function;

[0147] Said Optimized inverse of coupled network generators used to constrain global design properties.

[0148] Step (2): Utilize the local design attribute coupling network to extract the clothing image x with decorative design attributes from the source deco The decorative design attributes are disentangled and coupled with the global design attributes to output the clothing image with global design attributes. Fusion generates the final coupled image A fine-grained semantic segmentation network is used to extract the decorative or logo features of clothing images with decorative design attributes. Affine transformation is used to achieve strong positioning of decorative design attributes. The feature map of design attributes is learned through the encoder-bottleneck layer-decoder structure. The linear operation mask enables the network to have different attention, realize the natural transition between decorative design attributes and global design attributes, and output the final coupled image. And design the objective function of the local design attribute coupling network;

[0149] Step (2) is implemented through the following sub-steps:

[0150] (2.1) The source clothing image x has decorative design attributes deco Input the second decoupling module in the local design attribute coupling network for disentanglement to obtain a clothing image with decorative design attributes The source is a clothing image x with decorative design attributes deco The following conditions should be met: a)x deco It should be an image of clothing hanging on a hanger in a store or worn on a mannequin. After initial alignment, the clothing object should be centered in the image and have a high pixel ratio; b) x deco It should have a complete partial pattern or logo, and the decoration should have a clear outline and be easy to separate from the clothing object;

[0151] (2.1.1) For the input source clothing image x with decorative design attributes deco Perform affine transformation to obtain T·x deco , and then use a fine-grained semantic segmentation network or manual segmentation to obtain the segmentation map x′ deco , and T·x deco With the segmentation map x′ deco Perform element-wise multiplication to obtain clothing images with decorative design attributes Right now Where T∈R 3×3 is an affine matrix, a1~a4 are rotation and scaling coefficients; t1 and t2 are translation coefficients.

[0152] (2.2) Figure 4 As shown, the clothing image with decorative design attributes Garment images with global design attributes Clothing Image Mask and the partition map x′ deco The generator in the second coupling module in the local design attribute coupling network is input to obtain the final coupling image The generator is an encoder-bottleneck layer-decoder structure;

[0153] (2.2.1) Using the segmentation graph x′ deco and clothing image mask Strengthen the attention of the local design attribute coupling network to the overall contour and local decoration of the clothing; define the attention mask x′ that does not belong to the decorative part in the clothing object d ' eco as follows:

[0154]

[0155] Multiply This is to prevent the image from belonging to x′ deco But not The part causes threshold leakage.

[0156] (2.2.2) Clothing images with global design attributes Input encoder E glob Get the feature map η0, and transform the clothing image with decorative design attributes Input encoder E deco Get the feature map μ0, Among them, H1, W1 and C1 are the height, width and number of channels of the feature map, and the value of the number of channels C1 is 128;

[0157] (2.2.3) Attention mask x″ for the non-decorative parts of the clothing object deco Downsampling obtains the downsampling function γ(x′ deco ); for the segmentation graph x′ deco Downsampling obtains the downsampling function γ(x′ deco );Mask the clothing image Downsampling to get the downsampling function

[0158] (2.2.4) The feature map η0, feature map μ0, downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Input bottleneck layer 3, the bottleneck layer includes 2×3 bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; the feature map η0 is obtained by passing through the bottleneck layer to obtain η1, and the feature map μ0 is obtained by passing through the bottleneck layer to obtain μ1, and the downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Coupling is performed to obtain η′1: Where ⊙ is the element product symbol, b1 and b2 are the superposition coefficient and substitution coefficient respectively; η′1 is coupled again to obtain η′2: η′2 is coupled again to obtain η′3:

[0159] (2.2.5) Finally, η′3 is input into the decoder to output the final coupled image The final coupled image It realizes the extraction of clothing images with decorative design attributes. Garment images with global design attributes Clothing Image Mask and the partition map x′ deco Get geometry, texture, shading, and decorative design properties from .

[0160] (2.3) The design objective function of the local design attribute coupling network is the overall objective function of the generator design in the local design attribute coupling network:

[0161] (2.3.1) First, design the local L1 loss objective function; transform the clothing image with global design attributes and clothing images with decorative design attributes As the label image, the final coupled image As the generated result, a local L1 loss objective function is constructed The local L1 loss objective function is defined as follows:

[0162]

[0163] In the formula, λ glob ,λ loca is the corresponding weight coefficient;

[0164] (2.3.2) Secondly, we design the local L2 loss objective function: In order to make the clothing image with global design attributes and clothing images with decorative design attributes The transition between them is natural, using the local L2 loss objective function To blur the boundary, the local L2 loss objective function The definition of

[0165]

[0166] In the formula, λ glob ,λ loca is the corresponding weight coefficient;

[0167] (2.3.3) Finally, the local L1 loss objective function and the local L2 loss objective function are used to design the total objective function of the generator in the local design attribute coupling network: the total objective function L of the generator in the local design attribute coupling network LDAis defined as follows:

[0168]

[0169] Where λ6 and λ7 are the hyperparameters of the loss function;

[0170] The L LDA Optimization inverse of coupled network generators used to constrain local design properties.

[0171] Example 2

[0172] The embodiment of the present invention collects a clothing design attribute data set, clusters the data set, and uses it for training and testing a global design attribute coupling network and a local design attribute coupling network.

[0173] The training set of the clothing design attribute dataset consists of two parts: {x real} and {x deco}.

[0174] {x real} includes 2579 store top clothing images. The image size is 192×256, and the images are initially aligned to meet the high pixel ratio condition max(ρ w ,ρ h )=0.95, where ρ w , h The width and height ratios are respectively. The mask of the clothing image is obtained using manual segmentation Obtaining repetitive texture regions through manual annotation

[0175] {x deco} includes 1611 store top clothing images, the image size is also 192×256, and its mask x′ is obtained through manual segmentation deco .

[0176] The clothing design attribute dataset includes three test sets: test set 1, test set 2, and test set 3. Each test set includes {x shap}、{x text} and {x deco The images in the test sets come from different sources. The clothing images in test set 1 come from the same online clothing website, the clothing images in test set 2 come from different brands on online shopping websites, and the clothing images in test set 3 are clothing images.

[0177] The present invention uses the Adam optimizer to optimize network training, with a learning rate of 5×10 -4 During training, the batch size is 1 and the number of iterations is 150.

[0178] Generate the global design attribute coupling network and the local design attribute coupling network on test set 1, test set 2 and test set 3 and like Figure 5 As shown in , in test set 1, the clothing images have similar styles, and both the global design attribute coupling network and the local design attribute coupling network perform the task well. Figure 6 As shown in Figure 2, in test set 2, the styles, states, and shooting environments of clothing images are very different, but our method is still robust to such large-span domain transfer, because the shadows caused by ambient light in the test set 2 samples are more obvious, and the generated clothing images look more realistic. Figure 7 As shown in Figure 3, in test set 3, the clothing images are not aligned or are clothing images. Under extreme input conditions, our method can still generate clothing images with certain quality.

Claims

1. A method for generating clothing images based on disentangled representation of design attributes, characterized in that: The following steps are involved: Step (1): Use the global design attribute coupling network to obtain clothing images x from the store real Disentangle geometry, texture and shading design attributes and couple them to generate clothing images with global design attributes The global design attributes of clothing images in stores are disentangled using semantic segmentation, repeated region labeling, and Laplacian operators. A generator consisting of an encoder-bottleneck layer-decoder structure is used to learn the global design attributes, and a mixture of multiplication and adders is used to couple the global design attributes again to output clothing images with global design attributes. And design objective function for global design attribute coupling network; Step (2): Utilize the local design attribute coupling network to extract the clothing image x with decorative design attributes from the source deco The decorative design attributes are disentangled and coupled with the global design attributes to output the clothing image with global design attributes. Fusion generates the final coupled image A fine-grained semantic segmentation network is used to extract the decorative or logo features of clothing images with decorative design attributes. Affine transformation is used to achieve strong positioning of decorative design attributes. The feature map of design attributes is learned through the encoder-bottleneck layer-decoder structure. The linear operation mask enables the network to have different attention, realize the natural transition between decorative design attributes and global design attributes, and output the final coupled image. And the objective function of the network design is coupled with local design attributes.

2. The method for generating clothing images based on disentangled representation of design attributes according to claim 1, characterized in that: The step (1) is specifically: (1.1) The clothing image x in the store real Input the first decoupling module in the global design attribute coupling network for disentanglement, the clothing image x in the store real can be regarded as clothing images x with geometric shape design attributes. shap , clothing image x with texture design attributes text or clothing image x with shadow design attributes shad , the clothing image x with geometric shape design attributes shap Clothing image mask obtained after unwrapping The clothing image x with texture design attributes text The texture image formed by the repetitive area of ​​the array is obtained by unwrapping The clothing image x with shadow design attributes shad After unwrapping, the Laplace image gradient is obtained (1.2) Mask the clothing image obtained in step (1.1) Texture image formed by the repeated areas of the array and the Laplacian image gradient Input the generator in the first coupling module in the global design attribute coupling network, and finally obtain the clothing image with global design attributes The generator is an encoder-bottleneck layer-decoder structure; (1.3) The clothing image with global design attributes obtained in step (1.2) The texture attribute discriminator D is input to the first coupling module in the global design attribute coupling network. text and shadow attribute discriminator D shad In the above equation, we get the first result matrix ψ. text and the second result matrix ψ shad ; ( 1.4) Designing the objective function for the global design attribute coupling network includes designing an adversarial loss function for the discriminator in the global design attribute coupling network and designing an overall objective function for the generator in the global design attribute coupling network.

3. The method for generating clothing images based on design attribute disentanglement representation according to claim 2, characterized in that: The step (1.1) is specifically: (1.1.1) For clothing images x with geometric design attributes shap Generating Clothing Image Masks Using Semantic Segmentation Network Masking in clothing images In the example, binary (1 or 0) is used to distinguish clothing objects from background; the clothing image mask Only geometric shape design attributes are extracted from clothing images, without texture and shadow design attributes; (1.1.2) Extracting clothing images x with texture design attributes by marking landmarks of repetitive texture regions text The texture design attribute of the array is to fill the texture image formed by the repeated texture area of ​​the array The size of x text Same; the texture image formed by the array repetitive area Only the texture design attributes are represented from the clothing image, and the geometric shape and shadow design attributes of the clothing image are not included; (1.1.3) The Laplacian operator is used to extract the clothing image x with shadow design attributes shad The shadow design attribute of the clothing image x with shadow design attribute is unwrapped shad The brightness gradient feature is obtained by first Laplacian image gradient The first Laplacian image gradient It only represents the brightness gradient features in the shadow design attributes of clothing images, and discards the geometric shape and texture design attributes of clothing images; The first Laplacian image gradient Using formula (1), we can calculate: Where g(·) is the color to grayscale image conversion function; is the convolution operator; k l is the Laplace convolution kernel, whose size is 3×3, and generates the second-order differential of the clothing image by calculating the grayscale gradient of the eight neighborhoods. Set the Laplacian convolution kernel k l The step size is 1, and the Laplacian convolution operation filling is set to 1; the first Laplacian image gradient The size of x shad are the same size; (1.1.4) Mask the clothing image generated in step (1.1.1) The texture image formed by the array repetitive area obtained in step (1.1.2) First, perform element-wise multiplication to obtain a clothing image x with geometric shape and texture design attributes. fake ,Right now In the formula, ⊙ is the symbol of element product; then fake Use the Laplacian operator to extract the shadow design attributes and obtain the second Laplacian image gradient Right now (1.1.5) The first Laplacian image gradient obtained in step (1.1.3) and the second Laplacian image gradient Merge into Laplacian image gradient 4. The method for generating clothing images based on design attribute disentanglement representation according to claim 3, characterized in that: The step (1.2) is specifically: (1.2.1) Mask the clothing image Input geometry encoder E shap Get the geometric shape feature map Texture image formed by the repeated areas of the array Input texture encoder E text Get texture feature map Laplacian Image Gradient Input shadow encoder E shad Get the shadow feature map Said Where H1, W1 and C1 are the feature map height, width and number of channels, and the value of the channel number C1 is 128; the geometric shape encoder E shap , Texture Encoder E text and Shadow Encoder E shad have the same structure; (1.2.2) The geometric shape feature map Texture feature map and shadow feature map Parallel input bottleneck layer 1, the bottleneck layer 1 includes 3×3 bottleneck layers, each of which is composed of two {two-dimensional convolutional layers, instance normalization layers, activation layers} combined in series; geometric shape feature map Output after 3 bottleneck layers Texture feature map Output after 3 bottleneck layers Shadow feature map Output after 3 bottleneck layers In the gap between bottleneck layers, geometry, texture, and shading design properties are coupled again: in Where ⊙ is the element product; (1.2.3) Masking clothing images Downsampling to get the downsampling function The coupling result of step (1.2) Input bottleneck layer 2, where bottleneck layer 2 is composed of three bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; Coupling results Input the first bottleneck layer of bottleneck layer 2 to get and Coupling Then will Input the second bottleneck layer of bottleneck layer 2 to get Again with Coupling Then Input the third bottleneck layer of bottleneck layer 2 to get Again with Coupling (1.2.4) will eventually Input to the decoder outputs a clothing image with global design attributes Said Masking from clothing images Texture image formed by the repeated areas of the array and the Laplacian image gradient The geometry, texture, and shading design properties are coupled.

5. The method for generating clothing images based on disentangled representation of design attributes according to claim 4, characterized in that: The step (1.3) is specifically: (1.3.1) The clothing image with global design attributes obtained in step (1.3) Cropping to texture design attribute representation image Said The cropping area is as follows: {(x,y)∈R|x∈[Ο(x)-W2 / 2,Ο(x)+W2 / 2],y∈[Ο(y)-H2 / 2,Ο(y)+H2 / 2]} Where (x, y) is the texture design attribute representation image Pixel coordinates in the image; Ο(x) and Ο(y) are texture design attribute representation images The x and y coordinate values ​​of the center point; H2 and W2 are texture design attribute representation images The height and width of The texture design attributes are then used to represent the image Input to the texture attribute discriminator D text Generate the first result matrix ψ text , Where H3 and W3 are the first result matrix ψ text The height and width of the texture attribute discriminator D text It consists of four combined blocks {2D convolutional layer, batch normalization layer, leaky activation layer}; (1.3.2) Decoupling clothing images with global design attributes using the Laplacian operator The shadow design attribute of the image is generated Then the shadow design attribute represents the image Input shadow attribute discriminator D shad , and get the second result matrix ψ shad , Where H4 and W4 are the second result matrix ψ shad The height and width of the 6. The method for generating clothing images based on design attribute disentanglement representation according to claim 5, characterized in that: The step (1.4) is specifically: (1.4.1) Global Design Attributes Adversarial Loss Function for Discriminators in Coupled Networks: Texture Attribute Discriminator D text and shadow attribute discriminator D shad The adversarial loss function The definition is as follows: Where log[·] is the logarithmic function; Design attribute representation images for textures; Design attribute representation images for shadows; is the first Laplacian image gradient; is the second Laplace image gradient; E{·} is the expected value; x′ text To crop the label image The acquired image; Said Used to constrain the texture attribute discriminator D in the global design attribute coupling network text and shadow attribute discriminator D shad Optimized reverse; (1.4.2) The overall objective function design of the generator in the global design attribute coupling network: (1.4.2.1) Generator adversarial loss function: Design adversarial loss function for the generator in the coupled network with global design attributes Said is defined as follows: In the formula, is the weight coefficient; (1.4.2.2) Global L1 loss objective function: transform the clothing image x in the store real and an image x with geometric shape and texture design attributes fake As label images, clothing images with global design attributes As a result, a global L1 loss objective function is constructed Said is defined as follows: In the formula, is the weight coefficient; (1.4.2.3) VGG loss objective function: Use the pre-trained VGG-16 model to constrain the global design coupling network at the perception level, taking the clothing image x in the store real and an image x with geometric shape and texture design attributes fake As label images, clothing images with global design attributes Construct the VGG loss objective function as the generated result Said is defined as follows: In the formula, vgg i (·) is the i-th layer output of the VGG-16 model, with a total of 4 layers of output; and is the label image weight; x real and x fake is the label image; (1.4.2.4) Coupling consistency loss objective function: Assuming that the clothing image has global design attributes The unwrapped geometry and texture design properties remain x shap and x text , and the untangled shadow design properties are solved by the Laplacian operator; therefore, the coupled consistency loss objective function is defined as follows: In the formula, and is the weight coefficient of the label image, G GDA (·) is the output result of the generator in the global design attribute coupling network; (1.4.2.5) Gradient loss objective function: In order to further strengthen the coupling of shadow design attributes, the shadow design attribute representation image is constrained at full size The first Laplacian image gradient and the second Laplacian image gradient As the label image and compared with it, we get the gradient loss objective function Said is defined as follows: In the formula, and is the weight coefficient of the label image; (1.4.2.6) The total objective function of the generator in the global design attribute coupling network: The total objective function of the generator in the global design attribute coupling network is defined as follows: Where λ1, λ2, λ3, λ4 and λ5 are the hyperparameters of each loss function; Said Optimized inverse of coupled network generators used to constrain global design properties.

7. The method for generating clothing images based on design attribute disentanglement representation according to claim 1, characterized in that: The step (2) is specifically: (2.1) The source clothing image x has decorative design attributes deco Input the second decoupling module in the local design attribute coupling network for disentanglement to obtain a clothing image with decorative design attributes For the input source clothing image x with decorative design attributes deco Perform affine transformation to obtain T·x deco , and then use a fine-grained semantic segmentation network or manual segmentation to obtain the segmentation map x′ deco , and T·x deco With the segmentation map x′ deco Perform element-wise multiplication to obtain clothing images with decorative design attributes Right now Where T∈R 3×3 is an affine matrix, a1~a4 are rotation and scaling coefficients; t1 and t2 are translation coefficients; (2.2) Clothing images with decorative design attributes Garment images with global design attributes Clothing Image Mask and the partition map x′ deco The generator in the second coupling module in the local design attribute coupling network is input to obtain the final coupling image The generator is an encoder-bottleneck layer-decoder structure; (2.3) The design objective function of the local design attribute coupling network is the overall objective function of the generator design in the local design attribute coupling network.

8. The method for generating clothing images based on design attribute disentanglement representation according to claim 7, characterized in that: The step (2.2) is specifically: (2.2.1) Using the partition map x′ deco and clothing image mask Strengthen the attention of the local design attribute coupling network to the overall outline and local decoration of clothing; Define the attention mask x″ that does not belong to the decorative part in the clothing object deco as follows: Multiply This is to prevent the image from belonging to x′ deco But not The part causes threshold leakage; (2.2.2) Clothing images with global design attributes Input encoder E glob Get the feature map η0, and transform the clothing image with decorative design attributes Input encoder E deco Get the feature map μ0, Among them, H1, W1 and C1 are the height, width and number of channels of the feature map, and the value of the number of channels C1 is 128; (2.2.3) Attention mask x″ for the non-decorative parts of the clothing object deco Downsampling obtains the downsampling function γ(x′ deco ); for the segmentation graph x′ deco Downsampling obtains the downsampling function γ(x′ deco );Mask the clothing image Downsampling to get the downsampling function (2.2.4) The feature map η0, feature map μ0, downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Input bottleneck layer 3, the bottleneck layer includes 2×3 bottleneck layers, and the structure of each bottleneck layer is consistent with the bottleneck layer in bottleneck layer 1; the feature map η0 is obtained by passing through the bottleneck layer to obtain η1, and the feature map μ0 is obtained by passing through the bottleneck layer to obtain μ1, and the downsampling function γ(x′ deco ), downsampling function γ(x′ deco ) and downsampling function Coupling is performed to obtain η′1: Where ⊙ is the element product symbol, b1 and b2 are the superposition coefficient and substitution coefficient respectively; η′1 is coupled again to obtain η′2: η′2 is coupled again to obtain η′3: (2.2.5) Finally, η′3 is input into the decoder to output the final coupled image The final coupled image It realizes the extraction of clothing images with decorative design attributes. Garment images with global design attributes Clothing Image Mask and the partition map x′ deco Get geometry, texture, shading, and decorative design properties from .

9. The method for generating clothing images based on design attribute disentanglement representation according to claim 8, characterized in that: The step (2.3) is specifically: (2.3.1) First, design the local L1 loss objective function; transform the clothing image with global design attributes and clothing images with decorative design attributes As the label image, the final coupled image As the generated result, a local L1 loss objective function is constructed The local L1 loss objective function is defined as follows: In the formula, λ glob ,λ loca is the corresponding weight coefficient; (2.3.2) Secondly, we design the local L2 loss objective function: In order to make the clothing image with global design attributes and clothing images with decorative design attributes The transition between them is natural, using the local L2 loss objective function To blur the boundary, the local L2 loss objective function The definition of In the formula, λ glob ,λ loca is the corresponding weight coefficient; (2.3.3) Finally, the local L1 loss objective function and the local L2 loss objective function are used to design the total objective function of the generator in the local design attribute coupling network: the total objective function L of the generator in the local design attribute coupling network LDA is defined as follows: Where λ6 and λ7 are the hyperparameters of the loss function; The L LDA Optimization inverse of coupled network generators used to constrain local design properties.

Images