Method and device for one-key generation of dummy model clothing image, and storage medium

Abstract

The invention relates to the field of image processing, and discloses a method for one-key generation of a dummy model garment image, which comprises the following steps: S1, a user selects the gender of a garment: the user selects the gender (male or female) of the garment, and a system automatically allocates a corresponding dummy model according to the selection; according to the method for generating the dummy model clothing image through one key, the process of generating the image through AI is faster than that of traditional photography, a large amount of time is saved, model employment, photostudio building or logistics problem handling are not needed, the cost is greatly reduced, meanwhile, clothing and backgrounds can be easily replaced, different user requirements are met, and the user experience is improved. Compared with the method for generating the image by using a real person model, the method for generating the image by using the dummy model has the advantages that the portrait right of other people is not required to be invaded, the legality and privacy protection of the data can be completely ensured, and the size and the appearance of the dummy model are fixed, so that the consistency of the displayed image can be ensured.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method, a device and a storage medium for generating a clothing image of a dummy model with one click. Background Art

[0002] With the rapid development of e-commerce and online shopping, the clothing industry is facing increasingly fierce market competition, and consumers' requirements for shopping experience are also constantly increasing. In traditional clothing photography, shooting clothing images usually requires physical shooting in a studio with the help of cameras and real models. This method is not only time-consuming and labor-intensive, but also requires a lot of manpower and material costs. In addition, this traditional method is not flexible enough when it is necessary to change the shooting plan or try different clothing combinations. In this context, traditional clothing display methods can no longer meet the needs of brands and consumers;

[0003] Some existing solutions on the market often rely on complex production processes or expensive photography equipment, which not only consumes a lot of manpower, but may also cause delays due to the availability of models;

[0004] With the rapid development of artificial intelligence technology, especially its application in the field of image generation, AI has begun to emerge in the clothing industry. Although there have been attempts to use AI to generate images of clothing on models, these methods still have some limitations, such as insufficient realism of generated images, high difficulty in customization, and rights issues faced in generating real models. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies in the prior art, the present invention provides a method, device and storage medium for generating a clothing image of a mannequin with one click to solve the above problems.

[0007] (II) Technical solution

[0008] To achieve the above object, the present invention provides the following technical solution: a method for generating a clothing image of a mannequin with one key, comprising the following steps:

[0009] S1. User selects the gender of clothing: The user selects the gender of clothing (male or female), and the system automatically assigns the corresponding mannequin model according to the selection;

[0010] S2. Upload original clothing image: The user uploads the clothing image to be displayed;

[0011] S3. Extract clothing area image: The system uses the Grounding DINO algorithm to extract the preliminary clothing mask, uses the Segment Anything algorithm to refine the clothing mask, accurately extracts the clothing area, and performs pixel expansion to obtain the foreground image of the clothing area;

[0012] S4. Generate a clothing image of a mannequin: input the extracted clothing region image and the corresponding mannequin model into a preset neural network model to generate an image of a mannequin wearing a specified garment;

[0013] S5. Image post-processing: Use super-resolution reconstruction technology and detail enhancement algorithms to improve the clarity and realism of generated images;

[0014] S6. Display the final image: The system displays the generated image of the mannequin wearing the clothing for the user to review.

[0015] Preferably, the specific steps of using Grounding DINO to extract the preliminary clothing mask are:

[0016] Input preparation: provide an image containing clothing and a text prompt "clothing";

[0017] Model loading, load the pre-trained Grounding DINO model;

[0018] Inference process, images and text prompts are input into the model to perform reasoning;

[0019] Result acquisition, the model outputs a preliminary mask of the clothing area;

[0020] The Grounding DINO model combines the Visual Transformer (ViT) and the Transformer encoder-decoder architecture to achieve language and vision alignment. Its main components include:

[0021] Visual Backbone, which extracts high-level features of images based on visual Transformer or convolutional neural networks;

[0022] Transformer encoder, which processes text input and converts it into text embeddings;

[0023] Transformer decoder, which interacts visual features and text embeddings to generate the final detection and segmentation results;

[0024] The Grounding DINO model training process includes:

[0025] Self-supervised learning: pre-training on large-scale unlabeled datasets, using contrastive learning to enable the model to learn image representations without explicit labels;

[0026] Language-visual alignment, further trained on annotated datasets, learns the correspondence between text descriptions and image regions, using a contrastive loss function that enables the model to align the text prompt "clothing" with the corresponding region in the image.

[0027] Preferably, the specific steps of using Segment Anything to refine the clothing mask are:

[0028] Input preparation, using the preliminary mask generated by Grounding DINO as input;

[0029] Model loading, load Segment Anything model (SAM);

[0030] In the inference process, the preliminary mask is input into the SAM model for further segmentation;

[0031] Results obtained, the model outputs more accurate clothing masks;

[0032] The specific steps for extracting pixel values ​​of clothing areas are:

[0033] Mask application, using the refined mask to extract the pixel values ​​of the clothing area from the original image;

[0034] Foreground image generation, generating a foreground image containing only the clothing area;

[0035] The specific steps for pixel expansion are:

[0036] Edge detection, detecting the edges of clothing areas in the foreground image;

[0037] Pixel expansion: expands the pixels of the clothing area based on edge information to ensure smooth and natural edges;

[0038] Image interpolation uses image processing techniques to make the expanded pixels blend better with the background.

[0039] Preferably, the specific steps of neural network model training are:

[0040] Data preparation: using a large number of datasets containing clothing images and corresponding mannequin images, including different types of clothing and different backgrounds. Each pair of clothing images and mannequin images is labeled to ensure that the model learns the correct mapping relationship.

[0041] Model architecture, the image processing part uses a convolutional neural network (CNN) to extract the features of clothing images, and the generation part uses a generative adversarial network (GAN) or a diffusion model to generate the final mannequin clothing image;

[0042] During the training process, the model continuously optimizes its own parameters by comparing the differences between the generated images and the real images.

[0043] Preferably, the specific process of CNN extracting clothing image features is as follows: image standardization, adjusting the extracted clothing area image to the standardized size of the network input and performing normalization processing; data enhancement, improving the generalization ability of the model, performing data enhancement operations such as rotation, scaling, and cropping on the image; the first layer of convolution, using a small-sized convolution kernel to slide on the image to extract the low-level features of the image; activation function, applying the ReLU activation function to introduce nonlinearity and enhance the expression ability of the network; subsequent convolution layers, as the network depth increases, the size of the convolution kernel increases, or extracts by combining low-level features More advanced features; dimensionality reduction, reducing the size of the feature map through maximum pooling or average pooling; translation invariance, the pooling operation makes the features more translation invariant, that is, the features of the object can still be recognized when it moves in the image; multi-scale feature fusion, fusing multi-scale features through jump connections to enhance feature expression capabilities; automatically adjusting the importance of different channels through the channel attention mechanism to highlight key features; deep convolution, in the deep layer of the network, the convolution layer extracts more abstract semantic features; global features, converting the feature map into a global feature vector through global pooling or fully connected layers to capture the style and shape information of the entire garment.

[0044] Preferably, the specific steps of generating an image using a generative adversarial network (GAN) are as follows: a generator receives an input vector containing clothing features and mannequin parameters, and the generator gradually converts the vector into an image through a multi-layer neural network, and finally generates an image of a mannequin wearing specified clothing, wherein a discriminator receives the generated image and a real image as input, and the discriminator determines whether the input image is from the generator or a real data set. Through continuous iterations, the generator gradually learns to generate more realistic images, while the discriminator improves its discrimination ability.

[0045] Preferably, the specific steps of the diffusion model to generate an image are: the diffusion model starts from the original image, gradually adds noise, and finally obtains Gaussian noise. The model learns the process of gradually recovering the image from the noise. In the generation process, the model starts from random noise, gradually denoises, and finally generates an image of a mannequin wearing specified clothing. In the denoising process, the model conditionally accepts clothing features and mannequin parameters as input to guide the generation process.

[0046] According to another aspect of an embodiment of the present invention, a device for generating a clothing image of a mannequin with one click is also provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the method as described in any one of claims 1 to 7 are implemented.

[0047] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, wherein the storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, the steps of the method as described in any one of claims 1 to 7 are implemented.

[0048] According to another aspect of an embodiment of the present invention, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions, which, when executed, enable a machine to perform the steps of the method according to any one of claims 1 to 7.

[0049] (III) Beneficial effects

[0050] Compared with the prior art, the present invention provides a method, device and storage medium for generating a mannequin clothing image with one click, which has the following beneficial effects:

[0051] This one-click method for generating clothing images on mannequins uses AI to generate images faster than traditional photography, saving a lot of time. There is no need to hire models, build studios, or deal with logistical issues, which greatly reduces costs. At the same time, clothing and backgrounds can be easily changed to meet the needs of different users. Compared with using real models to generate images, using mannequins never needs to consider infringing on the portrait rights of others, and the legitimacy and privacy protection of data can be fully guaranteed. In addition, the size and appearance of the mannequins are fixed, so the consistency of the display images can be ensured, avoiding visual differences caused by individual differences in models. Finally, generating clothing images on mannequins is faster and more efficient, and does not require complex neural network models or post-processing, thereby saving time again. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0054] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0055] The method for generating a clothing image of a mannequin with one click comprises the following steps:

[0056] S1. User selects the gender of clothing: The user selects the gender of clothing (male or female), and the system automatically assigns the corresponding mannequin model according to the selection;

[0057] S2. Upload original clothing image: The user uploads the clothing image to be displayed;

[0058] S3. Extract clothing area image: The system uses the Grounding DINO algorithm to extract the preliminary clothing mask, uses the Segment Anything algorithm to refine the clothing mask, accurately extracts the clothing area, and performs pixel expansion to obtain the foreground image of the clothing area;

[0059] S4. Generate a clothing image of a mannequin: input the extracted clothing region image and the corresponding mannequin model into a preset neural network model to generate an image of a mannequin wearing a specified garment;

[0060] S5. Image post-processing: Use super-resolution reconstruction technology and detail enhancement algorithms to improve the clarity and realism of generated images;

[0061] S6. Display the final image: The system displays the generated image of the mannequin wearing the clothing for the user to review.

[0062] It should be noted that the specific steps of using Grounding DINO to extract the preliminary clothing mask are:

[0063] Input preparation: provide an image containing clothing and a text prompt "clothing";

[0064] Model loading, load the pre-trained Grounding DINO model;

[0065] Inference process, images and text prompts are input into the model to perform reasoning;

[0066] Result acquisition, the model outputs a preliminary mask of the clothing area;

[0067] The Grounding DINO model combines the Visual Transformer (ViT) and the Transformer encoder-decoder architecture to achieve language and vision alignment. Its main components include:

[0068] Visual Backbone, which extracts high-level features of images based on visual Transformer or convolutional neural networks;

[0069] Transformer encoder, which processes text input and converts it into text embeddings;

[0070] Transformer decoder, which interacts visual features and text embeddings to generate the final detection and segmentation results;

[0071] The Grounding DINO model training process includes:

[0072] Self-supervised learning: pre-training on large-scale unlabeled datasets, using contrastive learning to enable the model to learn image representations without explicit labels;

[0073] Language-visual alignment, further trained on annotated datasets, learns the correspondence between text descriptions and image regions, using a contrastive loss function that enables the model to align the text prompt "clothing" with the corresponding region in the image.

[0074] It should be noted that the specific steps of using Segment Anything to refine the clothing mask are:

[0075] Input preparation, using the preliminary mask generated by Grounding DINO as input;

[0076] Model loading, load Segment Anything model (SAM);

[0077] In the inference process, the preliminary mask is input into the SAM model for further segmentation;

[0078] Results obtained, the model outputs more accurate clothing masks;

[0079] The specific steps for extracting pixel values ​​of clothing areas are:

[0080] Mask application, using the refined mask to extract the pixel values ​​of the clothing area from the original image;

[0081] Foreground image generation, generating a foreground image containing only the clothing area;

[0082] The specific steps for pixel expansion are:

[0083] Edge detection, detecting the edges of clothing areas in the foreground image;

[0084] Pixel expansion: expands the pixels of the clothing area based on edge information to ensure smooth and natural edges;

[0085] Image interpolation uses image processing techniques to make the expanded pixels blend better with the background.

[0086] It should be noted that the specific steps of neural network model training are:

[0087] Data preparation: using a large number of datasets containing clothing images and corresponding mannequin images, including different types of clothing and different backgrounds. Each pair of clothing images and mannequin images is labeled to ensure that the model learns the correct mapping relationship.

[0088] Model architecture, the image processing part uses a convolutional neural network (CNN) to extract the features of clothing images, and the generation part uses a generative adversarial network (GAN) or a diffusion model to generate the final mannequin clothing image;

[0089] During the training process, the model continuously optimizes its own parameters by comparing the differences between the generated images and the real images.

[0090] It should be noted that the specific process of CNN extracting clothing image features is as follows: image standardization, adjusting the extracted clothing area image to the standardized size of the network input (256x256 pixels), and performing normalization processing; data enhancement, improving the generalization ability of the model, rotating, scaling, cropping and other data enhancement operations on the image; the first layer of convolution, using a small-sized convolution kernel (3x3) to slide on the image to extract the low-level features of the image (edges, colors); activation function, applying the ReLU activation function to introduce nonlinearity and enhance the expression ability of the network; subsequent convolution layers, as the network depth increases, the size of the convolution kernel increases, or by combining low-level features to extract higher-level features (texture, pattern); dimensionality reduction, reducing the size of the feature map through maximum pooling or average pooling; translation invariance, the pooling operation makes the features more translation invariant, that is, the features of the object can still be recognized when it moves in the image; multi-scale feature fusion, fusing multi-scale features through jump connections (residual networks) to enhance feature expression capabilities; automatically adjusting the importance of different channels through the channel attention mechanism (SE block) to highlight key features; deep convolution, in the deep layer of the network, the convolution layer extracts more abstract semantic features (local structure of clothes, sleeves, neckline); global features, converting feature maps into global feature vectors through global pooling or fully connected layers to capture the style and shape information of the entire garment.

[0091] It should be noted that the specific steps of generating images using a generative adversarial network (GAN) are as follows: the generator receives an input vector containing clothing features and mannequin parameters, and the generator gradually converts the vector into an image through a multi-layer neural network, and finally generates an image of a mannequin wearing specified clothing. The discriminator receives the generated image and the real image as input, and the discriminator determines whether the input image comes from the generator or the real data set. Through continuous iterations, the generator gradually learns to generate more realistic images, while the discriminator improves its discrimination ability.

[0092] It should be noted that the specific steps of the diffusion model to generate images are as follows: the diffusion model starts from the original image, gradually adds noise, and finally obtains Gaussian noise. The model learns to gradually restore the image from the noise. In the generation process, the model starts from random noise, gradually denoises, and finally generates an image of a mannequin wearing specified clothing. In the denoising process, the model conditionally accepts clothing features and mannequin parameters as input to guide the generation process.

[0093] According to another aspect of an embodiment of the present invention, a device for generating a clothing image of a mannequin with one click is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of any one of the methods described in claims 1 to 7 are implemented.

[0094] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the steps of any one of the methods described in claims 1 to 7 are implemented.

[0095] According to another aspect of an embodiment of the present invention, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions, which, when executed, cause a machine to perform the steps of the method according to any one of claims 1 to 7.

[0096] In summary, the present invention provides a method for generating clothing images of mannequins with one click. The process of generating images through AI is faster than traditional photography, which saves a lot of time. There is no need to hire models, build studios or deal with logistical issues, which greatly reduces costs. At the same time, clothing and backgrounds can be easily changed to meet the needs of different users. Compared with using real models to generate images, the use of mannequins never needs to consider infringing on the portrait rights of others, and the legitimacy and privacy protection of data can be fully guaranteed. In addition, the size and appearance of the mannequins are fixed, so the consistency of the display pictures can be ensured, and visual differences caused by individual differences of models can be avoided. Finally, generating clothing images of mannequins is faster and more efficient, and does not require complex neural network models or post-processing, thereby saving time again.

[0097] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating a clothing image of a mannequin with one click, characterized in that: The following steps are involved: S1. User selects the gender of the clothing: The user selects the gender of the clothing, and the system automatically assigns the corresponding mannequin model according to the selection; S2. Upload original clothing image: The user uploads the clothing image to be displayed; S3. Extract clothing area image: The system uses the Grounding DINO algorithm to extract the preliminary clothing mask, uses the Segment Anything algorithm to refine the clothing mask, accurately extracts the clothing area, and performs pixel expansion to obtain the foreground image of the clothing area; S4. Generate a clothing image of a mannequin: input the extracted clothing region image and the corresponding mannequin model into a preset neural network model to generate an image of a mannequin wearing a specified garment; S5. Image post-processing: Use super-resolution reconstruction technology and detail enhancement algorithms to improve the clarity and realism of generated images; S6. Display the final image: The system displays the generated image of the mannequin wearing the clothing for the user to review.

2. The method for generating a mannequin clothing image with one click according to claim 1, characterized in that: The specific steps of using Grounding DINO to extract preliminary clothing masks are: Input preparation, provide an image containing clothing and a text prompt "clothing"; Model loading, load the pre-trained Grounding DINO model; Inference process, images and text prompts are input into the model to perform reasoning; Result acquisition, the model outputs a preliminary mask of the clothing area; The Grounding DINO model combines the Visual Transformer (ViT) and the Transformer encoder-decoder architecture to achieve language and vision alignment. Its main components include: Visual Backbone, which extracts high-level features of images based on visual Transformer or convolutional neural networks; Transformer encoder, which processes text input and converts it into text embeddings; Transformer decoder, which interacts visual features and text embeddings to generate the final detection and segmentation results; The Grounding DINO model training process includes: Self-supervised learning: pre-training on large-scale unlabeled datasets, using contrastive learning to enable the model to learn image representations without explicit labels; Language-visual alignment, further trained on annotated datasets, learns the correspondence between text descriptions and image regions, using a contrastive loss function that enables the model to align the text prompt "clothing" with the corresponding region in the image.

3. The method for generating a mannequin clothing image with one click according to claim 1, characterized in that: The specific steps of using Segment Anything to refine the clothing mask are: Input preparation, using the preliminary mask generated by Grounding DINO as input; Model loading, load Segment Anything model (SAM); In the inference process, the preliminary mask is input into the SAM model for further segmentation; Results obtained, the model outputs more accurate clothing masks; The specific steps for extracting pixel values ​​of clothing areas are: Mask application, using the refined mask to extract the pixel values ​​of the clothing area from the original image; Foreground image generation, generating a foreground image containing only the clothing area; The specific steps for pixel expansion are: Edge detection, detecting the edges of clothing areas in the foreground image; Pixel expansion: expands the pixels of the clothing area based on edge information to ensure smooth and natural edges; Image interpolation uses image processing techniques to make the expanded pixels blend better with the background.

4. The method for generating a mannequin clothing image with one key according to claim 1, characterized in that: The specific steps of neural network model training are: Data preparation: using a large number of datasets containing clothing images and corresponding mannequin images, including different types of clothing and different backgrounds. Each pair of clothing images and mannequin images is labeled to ensure that the model learns the correct mapping relationship. Model architecture, the image processing part uses a convolutional neural network (CNN) to extract the features of clothing images, and the generation part uses a generative adversarial network (GAN) or a diffusion model to generate the final mannequin clothing image; During the training process, the model continuously optimizes its own parameters by comparing the differences between the generated images and the real images.

5. The method for generating a mannequin clothing image with one key according to claim 4, characterized in that: in, The specific process of CNN to extract clothing image features is as follows: image standardization, adjusting the extracted clothing area image to the standardized size of the network input and performing normalization processing; data enhancement, improving the generalization ability of the model, performing data enhancement operations such as rotation, scaling, and cropping on the image; The first convolution layer uses a small-sized convolution kernel to slide on the image to extract low-level features of the image; Activation function: Applying the ReLU activation function introduces nonlinearity and enhances the network's expressiveness; in subsequent convolutional layers, as the network depth increases, the size of the convolution kernel increases, or higher-level features are extracted by combining low-level features; Dimensionality reduction: Reduce the size of the feature map through maximum pooling or average pooling; Translation invariance: Pooling operations make features more translation invariant, that is, objects can still be recognized when they move in the image Its characteristics: Multi-scale feature fusion, fusing multi-scale features through jump connections to enhance feature expression capabilities; automatically adjusting the importance of different channels through the channel attention mechanism to highlight key features; deep convolution, in the deep layer of the network, the convolution layer extracts more abstract semantic features; global features, converting feature maps into global feature vectors through global pooling or fully connected layers to capture the style and shape information of the entire garment.

6. The method for generating a mannequin clothing image with one key according to claim 4, characterized in that: in, The specific steps of generating images using a generative adversarial network (GAN) are as follows: the generator receives an input vector containing clothing features and mannequin parameters, and gradually converts this vector into an image through a multi-layer neural network, and finally generates an image of a mannequin wearing specified clothing. The discriminator receives the generated image and the real image as input, and determines whether the input image comes from the generator or the real data set. Through continuous iterations, the generator gradually learns to generate more realistic images, while the discriminator improves its discrimination ability.

7. The method for generating a mannequin clothing image with one key according to claim 4, characterized in that: The specific steps of the diffusion model to generate images are as follows: the diffusion model starts from the original image, gradually adds noise, and finally obtains Gaussian noise. The model learns to gradually restore the image from the noise. In the generation process, the model starts from random noise, gradually denoises, and finally generates an image of a mannequin wearing specified clothing. In the denoising process, the model conditionally accepts clothing features and mannequin parameters as input to guide the generation process.

8. A device for generating a clothing image of a mannequin with one click, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when calling the computer program in the memory.

9. A computer-readable medium, characterized in that The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product is tangibly stored on a non-transitory computer readable medium and comprises machine executable instructions which, when executed, cause a machine to perform the steps of the method according to any one of claims 1 to 7.