Method for training a garment generation model and method for generating a garment

Abstract

Embodiments of the present disclosure provide a method for training a clothing generation model and a method for generating clothing. In the method for training the clothing generation model, a training image set of clothing items is obtained. The training image set includes a plurality of training image groups. For a training image group including a first and a second training image: color, texture and shape features of the first and the second training image are extracted; color, texture and shape verification images are generated by a clothing generator; a color and texture loss function is calculated according to the color and texture verification images and the first training image; a reconstruction loss function is calculated according to the shape verification image and the second training image; authenticity of the color, texture and shape verification images is judged by a real-fake discriminator; and parameters of the clothing generation model are updated by minimizing a total loss function of the clothing generation model. The total loss function includes: the color loss function, the texture loss function, the reconstruction loss function and a loss function of the real-fake discriminator.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically, to methods for training clothing generation models and methods for generating clothing. Background Technology

[0002] Color, texture, and shape constitute the basic elements of a garment. Color conveys visual characteristics and expresses the first impression of fashionable clothing. Texture refers to the quality of the fabric, the tactile feel of the garment. Shape, also known as the silhouette, can express the basic category of a garment and also convey information about the wearer's body type. The choice of color, texture, and shape for specific clothing can express the wearer's personality, character, and taste. Furthermore, fashion items can convey different effects through combinations of different colors, textures, and shapes. For example, a long red silk dress exudes luxury, while a red cotton skirt looks vibrant. In daily life, the color, texture, and shape of fashion items are also important factors considered by customers when purchasing. However, with the trend of large-scale industrial production of clothing today, factories find it difficult to produce fashion products with customized colors, textures, and shapes according to users' personalized requirements and preferences. Moreover, ordinary users lack design expertise and find it difficult to independently design fashion clothing according to their own preferences. Fortunately, the development of deep learning technology provides powerful tools for this kind of customized design, making intelligent assisted design feasible.

[0003] Existing generative adversarial networks (GANs) are powerful tools for generating clothing images. However, current clothing image design work mainly focuses on generating images using existing input conditions, such as labeled text information, clothing sketches, and clothing textures. Training these existing conditional models requires a large amount of labeled data, making them heavily dependent on the quality and quantity of labeled data. Therefore, there is a need to develop an unsupervised network that does not require any labeled information to adaptively disentangle and complete clothing design. Summary of the Invention

[0004] The embodiments described herein provide a method and apparatus for training a clothing generation model, a method and apparatus for generating clothing, and a computer-readable storage medium storing a computer program.

[0005] According to a first aspect of this disclosure, a method for training an apparel generation model is provided. The method includes: acquiring a training image set of individual apparel items, wherein the training image set includes multiple training image groups, each training image group including a first training image and a second training image; in each training round, performing the following operations for each training image group: extracting color features, texture features, and shape features from the first and second training images in the training image group using an attribute encoder in the apparel generation model; generating a color verification image based on the color features of the first training image and the texture and shape features of the second training image using an apparel generator in the apparel generation model; generating a texture verification image based on the color features and texture features of the first training image and the shape features of the second training image; and generating a shape verification image based on the color features, texture features, and shape features of the second training image. The process involves: verifying the shape of the image; calculating a color loss function based on the color similarity between the color verification image and the first training image; calculating a texture loss function based on the texture similarity between the texture verification image and the first training image; calculating a reconstruction loss function based on the shape consistency between the shape verification image and the second training image; determining the authenticity of the color verification image, texture verification image, and shape verification image using a real / fake discriminator; updating the parameters of the clothing generation model by minimizing the total loss function of the clothing generation model, where the total loss function of the clothing generation model includes: the color loss function, the texture loss function, the reconstruction loss function, and the loss function of the real / fake discriminator; and repeatedly training the clothing generation model until a predetermined number of training rounds are completed or the total loss function of the clothing generation model converges.

[0006] In some embodiments of this disclosure, calculating a color loss function based on the color similarity between a color calibration image and a first training image includes: using a filter to extract low-frequency components of the color calibration image and the first training image; and calculating a color loss function based on the similarity between the low-frequency components of the color calibration image and the low-frequency components of the first training image in the UV color space.

[0007] In some embodiments of this disclosure, the color loss function is calculated as follows:

[0008]

[0009] Where His(·) represents the histogram function of uv, a represents the first training image, m c This represents a color calibration image. This represents a low-frequency filtering function.

[0010] In some embodiments of this disclosure, calculating the texture loss function based on the texture similarity between the texture verification image and the first training image includes: using a filter to extract low-frequency components of the texture verification image and the first training image; subtracting the low-frequency components of the texture verification image from the texture verification image to obtain a texture representation of the texture verification image; subtracting the low-frequency components of the first training image from the first training image to obtain a texture representation of the first training image; randomly cropping the texture representation of the texture verification image to obtain a local texture representation of the texture verification image; randomly cropping the texture representation of the first training image to obtain a local texture representation of the first training image; performing feature encoding on the local texture representation of the texture verification image using a texture encoder to obtain local texture features of the texture verification image, and performing feature encoding on the local texture representation of the first training image to obtain local texture features of the first training image; and using a texture discriminator to determine the authenticity of the local texture features of the texture verification image and the local texture features of the first training image; wherein, the loss function of the texture discriminator is calculated as the texture loss function.

[0011] In some embodiments of this disclosure, the texture loss function is calculated as follows:

[0012]

[0013] Where a represents the first training image, m t Represents a texture verification image. E represents a random cropping operation. t D represents the texture encoder. t Represents a texture discriminator. This represents a low-frequency filtering function. This represents the training image set. This represents the set of verification images generated by the clothing generator.

[0014] In some embodiments of this disclosure, the reconstruction loss function is calculated as follows:

[0015]

[0016] Where b represents the second training image. Represents a shape verification image. This represents the training image set. This represents the set of verification images generated by the clothing generator.

[0017] In some embodiments of this disclosure, the loss function of the true / false discriminator is calculated as follows:

[0018]

[0019] in,

[0020]

[0021]

[0022] Where a represents the first training image, b represents the second training image, and m c Represents a color calibration image, m t Represents a texture verification image. Represents a shape verification image. This represents the training image set. Let D represent the set of verification images generated by the clothing generator, and let D represent the real / fake discriminator.

[0023] In some embodiments of this disclosure, the total loss function of the clothing generation model is calculated as follows:

[0024]

[0025] in, Represents the color loss function. Represents the texture loss function. Represents the reconstruction loss function. Let λ1, λ2, λ3, and λ4 represent the loss function of the true / false discriminator, where λ1, λ2, λ3, and λ4 are empirical values.

[0026] In some embodiments of this disclosure, the garment generator includes multiple cascaded garment pattern-making module levels. Each garment pattern-making module level includes two cascaded garment pattern-making modules and an upsampling module. Each garment pattern-making module includes a first affine transformation layer, a first modulation module, a first demodulation module, a first convolutional layer, a second affine transformation layer, a second modulation module, a second demodulation module, a second convolutional layer, and a stacking module. The first affine transformation layer is configured to perform linear transformation and translation operations on the input color features to generate a first intermediate vector. The first modulation module is configured to modulate the weights of the first intermediate vector to generate a first modulation vector. The first demodulation module is configured to normalize the weights of the first modulation vector. The first convolutional layer is configured to generate a first feature based on the normalized weights of the first modulation vector and a first input feature. The input shape feature is used as the first input feature of the first garment pattern-making module in the first garment pattern-making module level. The second affine transformation layer is configured to perform linear transformation and translation operations on the input texture features to generate a second intermediate vector. The second modulation module is configured to modulate the weights of the second intermediate vector to generate a second modulation vector. The second demodulation module is configured to normalize the weights of the second modulation vector. The second convolutional layer is configured to generate a second feature based on the normalized weights of the second modulation vector and a first feature. The superposition module is configured to superimpose the second feature and random noise to generate a superimposed feature. The superimposed feature is used as the first input feature of the next-level garment pattern-making module or as the input feature of the next-level upsampling module. The upsampling module is configured to upsample the superimposed feature output by its previous-level garment pattern-making module. The upsampled superimposed feature is used as the first input feature of the next-level garment pattern-making module.

[0027] According to a second aspect of this disclosure, an apparatus for training a clothing generation model is provided. The apparatus includes at least one processor and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the apparatus performs the steps of the method described according to a first aspect of this disclosure.

[0028] According to a third aspect of this disclosure, a computer-readable storage medium is provided storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described according to a first aspect of this disclosure.

[0029] According to a fourth aspect of this disclosure, a method for generating clothing is provided. The method includes: extracting color features of a color reference image, extracting texture features of a texture reference image, and extracting shape features of a shape reference image using an attribute encoder in a clothing generation model trained according to the method described in the first aspect of this disclosure; and generating a target clothing image based on the color features, texture features, and shape features using a clothing generator in the clothing generation model, such that the target clothing image has the color of the color reference image, the texture of the texture reference image, and the shape of the shape reference image.

[0030] According to a fifth aspect of this disclosure, an apparatus for generating clothing is provided. The apparatus includes at least one processor and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the apparatus causes to perform the steps of the method described according to a fourth aspect of this disclosure.

[0031] According to a sixth aspect of this disclosure, a computer-readable storage medium storing a computer program is provided, wherein the computer program, when executed by a processor, implements the steps of the method described according to a fourth aspect of this disclosure. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein:

[0033] Figure 1 This is an exemplary flowchart of a method for training a clothing generation model according to embodiments of the present disclosure;

[0034] Figure 2 This is an exemplary schematic diagram illustrating the process of training a clothing generation model according to an embodiment of the present disclosure;

[0035] Figure 3 This is an exemplary structural diagram of a clothing generator in a clothing generation model according to an embodiment of the present disclosure;

[0036] Figure 4 This is a schematic block diagram of an apparatus for training a clothing generation model according to an embodiment of the present disclosure;

[0037] Figure 5 This is an exemplary flowchart of a method for generating clothing according to embodiments of the present disclosure;

[0038] Figure 6 This is an exemplary relationship diagram between the reference image input to the clothing generation model and the target clothing image output from the clothing generation model; and

[0039] Figure 7 This is a schematic block diagram of an apparatus for generating clothing according to an embodiment of the present disclosure.

[0040] It should be noted that the elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having meanings consistent with their meanings in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. Furthermore, terms such as “first” and “second” are used only to distinguish one component (or part of a component) from another component (or another part of a component).

[0043] This disclosure proposes an unsupervised clothing generation model. This unsupervised clothing generation model may include an attribute encoder and a clothing generator. Figure 1 An exemplary flowchart illustrating a method for training a clothing generation model according to an embodiment of the present disclosure is shown. Figure 2 An exemplary schematic diagram illustrating the process of training a clothing generation model according to an embodiment of the present disclosure is shown below. Figure 1 and Figure 2 This describes the process of training a clothing generation model.

[0044] exist Figure 1 At frame S102, a training image set for each garment item is obtained. This training image set includes multiple training image groups. Each training image group includes a first training image and a second training image. Each training image is an image of a single garment item. Figure 2 In the example, the first training image is image a, and the second training image is image b. In some embodiments of this disclosure, any two images can be selected from the training image set to form a training image group.

[0045] In each round of training, the operations at boxes S104 to S120 are performed for each group of training images.

[0046] exist Figure 1 At box S104, the attribute encoder in the clothing generation model extracts the color features, texture features, and shape features of the first and second training images in the training image set. In some embodiments of this disclosure, the attribute encoder may be pre-trained. In some embodiments of this disclosure, the attribute encoder architecture is based on a residual network architecture for deep feature extraction of the training images, while encoding the training images into latent vector spaces of color, texture, and shape through a convolutional network to obtain the color features, texture features, and shape features of the training images. Figure 2 In the example, the attribute encoder 210 can encode the first training image a into a latent vector space of color, texture, and shape, which can be represented as: E(a) = {a c a t a s}. The color features of the first training image are determined by a. c To represent. The texture features of the first training image are represented by a t To represent. The shape features of the first training image are represented by a. s The attribute encoder 210 can encode the second training image b into a latent vector space of color, texture, and shape, which can be represented as: E(b) = {b c b t b s}. The color features of the second training image are determined by b. c To represent. The texture features of the second training image are represented by b. t To represent. The shape features of the second training image are represented by b. s To express.

[0047] exist Figure 1 At box S106, the clothing generator in the clothing generation model generates a color verification image based on the color features of the first training image and the texture and shape features of the second training image; generates a texture verification image based on the color and texture features of the first training image and the shape features of the second training image; and generates a shape verification image based on the color, texture, and shape features of the second training image. Figure 2 In the example, the color feature a of the first training image c The texture features b of the second training image t and shape features b s The input is used by the clothing generator 220 to generate a color calibration image m c The color features a of the first training image c and texture features a t and the shape features b of the second training image sThe image m is input to the clothing generator 220 to generate a texture verification image. t The color features b of the second training image c Texture features b t and shape features b s The image is input into the clothing generator 220 to generate a shape verification image.

[0048] Figure 3 An exemplary structural diagram of a clothing generator 320 in a clothing generation model according to an embodiment of the present disclosure is shown. Figure 3 As shown, the garment generator 320 may include multiple cascaded garment pattern-making module levels. Each garment pattern-making module level includes two cascaded garment pattern-making modules 320a and 320b and an upsampling module 320c. Each garment pattern-making module 320a and 320b includes a first affine transformation layer 321, a first modulation module 322, a first demodulation module 323, a first convolutional layer 324, a second affine transformation layer 325, a second modulation module 326, a second demodulation module 327, a second convolutional layer 328, and a stacking module 329.

[0049] Figure 3 This illustrates the color features a of the first training image input into the clothing generator 320. c and texture features a t and the shape features b of the second training image s To generate texture verification image m t Example. The first affine transformation layer 321 is configured to transform the input color feature a c A linear transformation and translation operation is performed on the vectors to generate the first intermediate vector. This affine transformation process can be represented as follows: Where, θ c Represents the first intermediate vector. This represents an affine transformation.

[0050] To better utilize color feature information, the first modulation module 322 is configured to modulate the first intermediate vector θ. c The weights are modulated to generate the first modulation vector. This modulation process can be represented as:

[0051]

[0052] in, Represents the first intermediate vector θ c The weight, The first modulation vector is represented by θ, where i represents the weight of the first intermediate vector. c The feature layer number; j represents the feature layer number of the convolution output; k represents the channel number of the convolution output; Indicates the modulation ratio. Figure 3 middle, Represents the first intermediate vector θ c The initial weights.

[0053] To better capture the style conveyed by the colors in the clothing image, the first modulation vector needs to be demodulated to its standard deviation. The first demodulation module 323 is configured to weight the first modulation vector... Normalization is performed:

[0054]

[0055] Where ∈ represents a constant (which is set as small as possible) to avoid the denominator in equation (2) being zero. This represents the weights after normalization, i.e., the convolution weights of the first convolutional layer 324.

[0056] The first convolutional layer 324 is configured according to the normalized weights of the first modulation vector. and the first input feature (the shape feature of the input b) s The first feature m1 is generated by using the first input feature of the first garment pattern making module in the first garment pattern making module level.

[0057]

[0058] in, Indicates the convolution operation; express The convolution weights.

[0059] Similarly, the second affine transformation layer 325 is configured to process the input texture features a t A linear transformation and translation operation is performed on the vector to generate a second intermediate vector. The second modulation module 326 is configured to modulate the weights of the second intermediate vector to generate a second modulation vector. The second demodulation module 327 is configured to normalize the weights of the second modulation vector.

[0060]

[0061] Where ∈ represents a constant (which is set as small as possible) to avoid the denominator in equation (4) being zero. This represents the normalized weights. The second convolutional layer 328 is configured with normalized weights based on the second modulation vector. The second feature m2 is generated from the first feature m1.

[0062]

[0063] in, This indicates the second convolutional layer, 328.

[0064] In some embodiments of this disclosure, the structure of the second affine transform layer 325 is the same as that of the first affine transform layer 321. The structure of the second modulation module 326 is the same as that of the first modulation module 322. The structure of the second demodulation module 327 is the same as that of the first demodulation module 323. The structure of the second convolutional layer 328 is the same as that of the first convolutional layer 324.

[0065] The overlay module 329 is configured to overlay the second feature m2 and random noise B to generate an overlay feature. The overlay feature output by the pattern-making module 320a is used as the first input feature to the next-level pattern-making module 320b. The overlay feature output by the pattern-making module 320b is used as the input feature to the next-level upsampling module 320c.

[0066] The upsampling module 320c is configured to upsample the superimposed features output by its parent garment pattern-making module 320b. The upsampled superimposed features are then used as the first input features to the next-level garment pattern-making module 320a. Figure 3 In the example, a 4×4 stacked feature can be upsampled to an 8×8 stacked feature.

[0067] In some embodiments of this disclosure, the garment generator 320 may include 7 cascaded garment pattern-making modules, i.e., 14 garment pattern-making modules 320a and 320b, so that the resolution of the output verification image is 256×256.

[0068] exist Figure 1 At box S108, the color loss function is calculated based on the color similarity between the color verification image and the first training image. Figure 2 In the example, it is necessary to fully utilize the color information of the first training image a without considering edge and texture details. In some embodiments of this disclosure, a filter can be used to extract the color calibration image m during the process of calculating the color loss function based on the color similarity between the color calibration image and the first training image. c The low-frequency components of the first training image a and the low-frequency components of the color calibration image m are extracted. c The processes of low-frequency components can be represented as follows: This represents a low-frequency filtering function.

[0069] However, the shapes of the low-frequency components in the images of different clothing items are not consistent. To remove the influence of different shapes, a UV histogram can be used to extract the low-frequency component a. gf and The color representation. The Hellinger distance between different images, calculated using a UV histogram, can be defined as follows:

[0070]

[0071] Where His(·) represents the histogram function of uv, a represents the first training image, m c This represents a color calibration image. This represents a low-frequency filtering function.

[0072] The above process can also be understood as verifying the low-frequency components of the image based on the color calibration in the UV color space. The low-frequency component a of the first training image gf The color loss function is calculated based on the similarity between the two values. The color loss function is calculated according to the expression in equation (6).

[0073] At box S110, a texture loss function is calculated based on the texture similarity between the texture verification image and the first training image. In this disclosure, texture refers to the texture details of the fabric of the garment item, without considering the color of the fabric.

[0074] In some embodiments of this disclosure, during the calculation of the texture loss function based on the texture similarity between the texture verification image and the first training image, a filter can be used to extract the low-frequency components of the texture verification image and the low-frequency components of the first training image. The structure of this filter can be the same as that of the filter used in the operation at block S108.

[0075] Then, the low-frequency components of the texture verification image are subtracted from the texture verification image to obtain the texture representation of the texture verification image. in, The texture representation of the texture verification image, m t Represents a texture verification image. This represents a low-frequency filtering function.

[0076] The low-frequency components of the first training image can be subtracted from the first training image in parallel to obtain the texture representation of the first training image. Where a represents the first training image, a sub This represents the texture representation of the first training image. This represents a low-frequency filtering function.

[0077] Due to texture verification image m t The shapes of the clothing items in the image may differ from those in the first training image a. If the entire texture verification image m is used... tWhen compared to the entire first training image 'a', the texture will be affected by the background in the image. Therefore, it is necessary to remove the texture from the verification image 'm'. t The influence of the background on the first training image a. This can be applied to the texture verification image m. t Texture representation Perform random cropping to obtain texture verification image m t The local texture representation of the first training image a, and the texture representation of the first training image a. sub Random cropping is performed to obtain a local texture representation of the first training image *a*. The location of the random cropping is constrained by pre-determined masks. Each mask only covers the location of the corresponding clothing item in the image. Then, the texture verification image *m* is compared. t The local texture representation of the first training image a and the local texture representation of the second training image a. Specifically, the texture encoder 230 can sequentially process the texture verification image m. t The local texture representation is used for feature encoding to obtain the texture verification image m. t The local texture features of the first training image a are obtained by feature encoding the local texture representation of the first training image a. Then, the texture discriminator 240 is used to determine the texture verification image m. t The authenticity of the local texture features of the texture discriminator 240 is compared with that of the local texture features of the first training image a. The loss function of the texture discriminator 240 is calculated as the texture loss function.

[0078] In some embodiments of this disclosure, the texture loss function is calculated as follows:

[0079]

[0080] Where a represents the first training image, m t Represents a texture verification image. E represents a random cropping operation. t D represents the texture encoder. t Represents a texture discriminator. This represents a low-frequency filtering function. Represents the training image set, This represents the set of verification images generated by the clothing generator.

[0081] At box S112, the reconstruction loss function is calculated based on the shape consistency between the shape verification image and the second training image. This is to ensure the shape verification image... To ensure shape consistency with the second training image b, a reconstruction loss can be used to learn the reconstruction process of the second training image b. In some embodiments of this disclosure, the reconstruction loss function is calculated as follows:

[0082]

[0083] Where b represents the second training image. Represents a shape verification image. Represents the training image set, This represents the set of verification images generated by the clothing generator.

[0084] At box S114, a true / false discriminator is used to determine the authenticity of the color verification image, texture verification image, and shape verification image.

[0085] To distinguish color calibration images m c To determine the authenticity of the color verification image m, embodiments of this disclosure use generative adversarial loss. c The optimization is performed, and the loss function is expressed as follows:

[0086]

[0087] in, Represents the training image set, Let m represent the set of verification images generated by the clothing generator, where a represents the first training image, and m represents the set of verification images generated by the clothing generator. c This represents a color verification image, and D represents a true / false discriminator.

[0088] To ensure the texture verification image m t To verify the authenticity of the texture image m, embodiments of this disclosure use a true / false discriminator D to perform texture verification. t The loss function is expressed as follows: The determination of whether a value is true or false is performed.

[0089]

[0090] Where a represents the first training image, m t Represents a texture verification image. Represents the training image set, This represents the set of verification images generated by the clothing generator, and D represents the real / fake discriminator.

[0091] To ensure the shape verification image To verify the authenticity of the image, embodiments of this disclosure use a true / false discriminator D to check the shape of the image. The authenticity is determined by the loss function, which is expressed as follows:

[0092]

[0093] Where b represents the second training image. Represents a shape verification image. Represents the training image set, This represents the set of verification images generated by the clothing generator, and D represents the real / fake discriminator.

[0094] Therefore, the loss function of the true / false discriminator is calculated as follows:

[0095]

[0096] At box S116, the parameters of the clothing generation model are updated by minimizing the total loss function of the clothing generation model. The total loss function of the clothing generation model includes: a color loss function, a texture loss function, a reconstruction loss function, and a loss function for the real / fake discriminator. In some embodiments of this disclosure, the total loss function of the clothing generation model is calculated as follows:

[0097]

[0098] in, Represents the color loss function. Represents the texture loss function. Represents the reconstruction loss function. Let λ1, λ2, λ3, and λ4 represent the loss function of the true / false discriminator, where λ1, λ2, λ3, and λ4 are empirical values.

[0099] In some embodiments of this disclosure, λ1 = 1, λ2 = 1, λ4 = 2.

[0100] At box S118, it is determined whether the total loss function of the clothing generation model has converged. If the total loss function does not decrease after several consecutive training rounds (e.g., 10 rounds), then the total loss function has converged. If the total loss function of the clothing generation model has converged ("Yes" at box S118), the process proceeds to box S122, and the training of the clothing generation model ends. If the total loss function of the clothing generation model has not converged ("No" at box S118), the process proceeds to box S120 to determine whether the predetermined number of training rounds has been completed. The predetermined number of training rounds can be set based on empirical values, estimated based on the expected training time, or considered based on the resource consumption of the computing device.

[0101] If the predetermined number of training rounds is not completed (marked as "No" at box S120), the process proceeds to box S104, and the clothing generation model is trained again. If the predetermined number of training rounds is completed (marked as "Yes" at box S120), the process proceeds to box S122, and the training of the clothing generation model ends.

[0102] In some embodiments of this disclosure, the clothing generation model can be implemented using a disentangled generative adversarial network. The training process of the clothing generation model described above can be used to form a color optimization path through equations (6) and (9), a texture optimization path through equations (7) and (10), and a shape optimization path through equations (8) and (11). By introducing three independent paths to extract the color, texture, and shape features of the clothing, the entire disentangled generative adversarial network is optimized in an unsupervised manner.

[0103] Figure 4 A schematic block diagram of an apparatus 400 for training a clothing generation model according to an embodiment of the present disclosure is shown. Figure 4 As shown, the device 400 may include a processor 410 and a memory 420 storing a computer program. When the computer program is executed by the processor 410, the device 400 is made capable of performing actions such as... Figure 1 The steps of method 100 are shown. In one example, device 400 may be a computer device or a cloud computing node. Device 400 can acquire a training image set of clothing items. The training image set includes multiple training image groups, each training image group including a first training image and a second training image. In each round of training, device 400 can perform the following operations for each training image group: extracting color features, texture features, and shape features of the first and second training images in the training image group by the attribute encoder in the clothing generation model; generating a color verification image based on the color features of the first training image and the texture and shape features of the second training image by the clothing generator in the clothing generation model; generating a texture verification image based on the color features and texture features of the first training image and the shape features of the second training image; and generating a shape verification image based on the color features, texture features, and shape features of the second training image; and determining the color consistency between the color verification image and the first training image. The color loss function is calculated based on similarity; the texture loss function is calculated based on the texture similarity between the texture verification image and the first training image; the reconstruction loss function is calculated based on the shape consistency between the shape verification image and the second training image; the authenticity of the color verification image, texture verification image, and shape verification image is determined by a real / fake discriminator; and the parameters of the clothing generation model are updated by minimizing the total loss function of the clothing generation model, where the total loss function of the clothing generation model includes: the color loss function, the texture loss function, the reconstruction loss function, and the loss function of the real / fake discriminator; and the clothing generation model is repeatedly trained until a predetermined number of training rounds are completed or the total loss function of the clothing generation model converges.

[0104] In embodiments of this disclosure, processor 410 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. Memory 420 may be any type of memory implemented using data storage technologies, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0105] Furthermore, in embodiments of this disclosure, the device 400 may also include an input device 430, such as a keyboard or mouse, for inputting a training image set. Additionally, the device 400 may also include an output device 440, such as a display, for outputting training results.

[0106] Figure 5 An exemplary flowchart of a method for generating clothing according to embodiments of the present disclosure is shown. Figure 5 At box S502, use according to Figure 1 The method shown trains an attribute encoder in the clothing generation model to extract color features from a color reference image, texture features from a texture reference image, and shape features from a shape reference image. In some embodiments of this disclosure, the color reference image and the texture reference image may be the same image, or the color reference image and the shape reference image may be the same image, or the texture reference image and the shape reference image may be the same image.

[0107] At box S504, the clothing generator in the clothing generation model is used to generate a target clothing image based on color features, texture features, and shape features, so that the target clothing image has the color of the color reference image, the texture of the texture reference image, and the shape of the shape reference image.

[0108] Figure 6 An exemplary relationship diagram is shown between the reference image input to the clothing generation model and the target clothing image output from the clothing generation model. Figure 6 The first row shows the color reference image, the second row shows the texture reference image, the third row shows the shape reference image, and the fourth row shows the target clothing image. Figure 6 Each column in the diagram represents a set of color reference images, texture reference images, shape reference images, and target clothing images. The clothing generation model generates the target clothing image based on the color reference images, texture reference images, and shape reference images in each column. For easier visualization, in... Figure 6 The first column shows an abstract image. In practical use, clothing generation models are mainly used to generate... Figure 6 The target clothing images are shown in columns two through six.

[0109] Figure 7A schematic block diagram of an apparatus 700 for generating clothing according to an embodiment of the present disclosure is shown. Figure 7 As shown, the device 700 may include a processor 710 and a memory 720 storing a computer program. When the computer program is executed by the processor 710, the device 700 is made capable of performing actions such as... Figure 5 The steps of method 500 are shown. In one example, device 700 may be a computer device or a cloud computing node. Device 700 may use according to... Figure 1 The method shown trains an attribute encoder in the clothing generation model to extract color features from a color reference image, texture features from a texture reference image, and shape features from a shape reference image. The device 700 can use... Figure 1 The clothing generator in the clothing generation model trained by the method shown generates a target clothing image based on color features, texture features, and shape features, so that the target clothing image has the color of the color reference image, the texture of the texture reference image, and the shape of the shape reference image.

[0110] In embodiments of this disclosure, processor 710 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. Memory 720 may be any type of memory implemented using data storage technologies, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0111] Furthermore, in embodiments of this disclosure, the device 700 may also include an input device 730, such as a keyboard or mouse, for inputting color reference images, texture reference images, and shape reference images. Additionally, the device 700 may also include an output device 740, such as a display, for outputting target clothing images.

[0112] In other embodiments of this disclosure, a computer-readable storage medium storing a computer program is also provided, wherein the computer program, when executed by a processor, is capable of performing the following functions: Figure 1 or Figure 5 The steps of the method shown.

[0113] In summary, the method for training a clothing generation model according to embodiments of this disclosure can optimize the unentangled generative adversarial network for color, texture, and shape in an unsupervised manner, avoiding the clothing generation model's heavy dependence on the quality and quantity of labeled data. The method for generating clothing according to embodiments of this disclosure can help users controllably complete the design of diverse clothing items, reducing the workload for users when designing clothing items. The clothing generation model according to embodiments of this disclosure has great potential in practical application areas such as controllable fashion design and interactive design with users.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses and methods according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0115] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the terms “comprising” and “including” shall be interpreted as including rather than exclusively. Likewise, the terms “including” and “or” shall be interpreted as including unless such interpretation is expressly prohibited herein. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or extensive.

[0116] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0117] Several embodiments of this disclosure have been described in detail above. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of this disclosure without departing from the spirit and scope of this disclosure. The scope of protection of this disclosure is defined by the appended claims.

Claims

1. A method for training a clothing generation model, comprising: Obtain a training image set for clothing items, wherein the training image set includes multiple training image groups, and each training image group includes a first training image and a second training image; In each round of training, perform the following operations for each group of training images: The attribute encoder in the clothing generation model extracts the color features, texture features, and shape features of the first training image and the second training image in the training image set; The clothing generator in the clothing generation model generates a color verification image based on the color features of the first training image and the texture and shape features of the second training image, generates a texture verification image based on the color and texture features of the first training image and the shape features of the second training image, and generates a shape verification image based on the color, texture and shape features of the second training image. The color loss function is calculated based on the color similarity between the color verification image and the first training image. The texture loss function is calculated based on the texture similarity between the texture verification image and the first training image. The reconstruction loss function is calculated based on the shape consistency between the shape verification image and the second training image; The authenticity of the color verification image, texture verification image, and shape verification image is determined using a real / fake detector; and The parameters of the clothing generation model are updated by minimizing the total loss function of the clothing generation model, wherein the total loss function of the clothing generation model includes: the color loss function, the texture loss function, the reconstruction loss function, and the loss function of the real / fake discriminator; and The clothing generation model is trained repeatedly until a predetermined number of training rounds are completed or the total loss function of the clothing generation model converges.

2. The method according to claim 1, wherein, Calculating the color loss function based on the color similarity between the color verification image and the first training image includes: A filter is used to extract the low-frequency components of the color calibration image and the low-frequency components of the first training image; and The color loss function is calculated based on the similarity between the low-frequency components of the color verification image and the low-frequency components of the first training image in the UV color space.

3. The method according to claim 2, wherein, The color loss function is calculated as follows: Where His(·) represents the histogram function of uv, a represents the first training image, m c This refers to the color verification image. This represents a low-frequency filtering function.

4. The method according to claim 1, wherein, Calculating the texture loss function based on the texture similarity between the texture verification image and the first training image includes: A filter is used to extract the low-frequency components of the texture verification image and the low-frequency components of the first training image. The low-frequency component of the texture verification image is subtracted from the texture verification image to obtain the texture representation of the texture verification image; Subtract the low-frequency components of the first training image from the first training image to obtain the texture representation of the first training image; The texture representation of the texture verification image is randomly cropped to obtain a local texture representation of the texture verification image; The texture representation of the first training image is randomly cropped to obtain the local texture representation of the first training image; The local texture features of the texture verification image are obtained by performing feature encoding on the local texture representation of the texture verification image through a texture encoder, and the local texture features of the first training image are obtained by performing feature encoding on the local texture representation of the first training image. The authenticity of the local texture features of the texture verification image and the local texture features of the first training image is determined by a texture discriminator. The loss function of the texture discriminator is calculated as the texture loss function.

5. The method according to claim 4, wherein, The texture loss function is calculated as follows: Where a represents the first training image, m t This represents the texture verification image. E represents a random cropping operation. t D represents the texture encoder. t This refers to the texture discriminator. This represents a low-frequency filtering function. This represents the training image set. This represents the set of verification images generated by the clothing generator.

6. The method according to claim 1, wherein, The reconstruction loss function is calculated as follows: Where b represents the second training image, This represents the shape verification image. This represents the training image set. This represents the set of verification images generated by the clothing generator.

7. The method according to claim 1, wherein, The loss function of the true / false discriminator is calculated as follows: in, Where a represents the first training image, b represents the second training image, and m c This represents the color calibration image, m t Represents a texture verification image. This represents the shape verification image. This represents the training image set. Let D represent the set of verification images generated by the clothing generator, and let D represent the real / fake discriminator.

8. The method according to claim 1, wherein, The total loss function of the clothing generation model is calculated as follows: in, This represents the color loss function. This represents the texture loss function. This represents the reconstruction loss function. Let λ1, λ2, λ3, and λ4 represent the loss function of the true / false discriminator, where λ1, λ2, λ3, and λ4 are empirical values.

9. The method according to claim 1, wherein, The garment generator comprises multiple cascaded garment pattern-making modules. Each garment pattern-making module includes two cascaded garment pattern-making modules and an upsampling module. Each garment pattern-making module includes a first affine transform layer, a first modulation module, a first demodulation module, a first convolutional layer, a second affine transform layer, a second modulation module, a second demodulation module, a second convolutional layer, and a stacking module. The first affine transformation layer is configured to perform linear transformation and translation operations on the input color features to generate a first intermediate vector. The first modulation module is configured to modulate the weights of the first intermediate vector to generate a first modulation vector; The first demodulation module is configured to normalize the weights of the first modulation vector; The first convolutional layer is configured to generate a first feature based on the normalized weights of the first modulation vector and the first input feature, wherein the shape feature of the input is used as the first input feature of the first garment pattern making module in the first garment pattern making module level. The second affine transformation layer is configured to perform linear transformation and translation operations on the input texture features to generate a second intermediate vector; The second modulation module is configured to modulate the weights of the second intermediate vector to generate a second modulation vector; The second demodulation module is configured to normalize the weights of the second modulation vector; The second convolutional layer is configured to generate a second feature based on the normalized weights of the second modulation vector and the first feature; The overlay module is configured to overlay the second feature and random noise to generate an overlay feature, wherein the overlay feature is used as the first input feature of the next-level garment pattern-making module or the input feature of the next-level upsampling module. The upsampling module is configured to perform an upsampling operation on the superimposed features output by the garment pattern-making module at the next higher level, wherein the superimposed features after the upsampling operation are used as the first input features for the garment pattern-making module at the next higher level.

10. A method for generating clothing, comprising: The attribute encoder in the clothing generation model trained according to any one of claims 1 to 9 is used to extract color features from the color reference image, texture features from the texture reference image, and shape features from the shape reference image. as well as The clothing generator in the clothing generation model is used to generate a target clothing image based on the color features, the texture features, and the shape features, so that the target clothing image has the color of the color reference image, the texture of the texture reference image, and the shape of the shape reference image.

Images