Image generation method, apparatus, device, and storage medium

Abstract

The application discloses an image generation method and device, equipment and a storage medium, and relates to the technical field of picture processing. The method comprises the following steps: acquiring a current clothing image and a current model image; inputting the current clothing image and the current model image into a preset image generation model, wherein the preset image generation model is constructed based on a diffusion model and is trained through wavelet transform; extracting clothing features of the current clothing image through the preset image generation model, and performing feature fusion on the clothing features and the current model image to obtain a model clothing material picture. Compared with the existing process of relying on a photography team to shoot and manually processing pictures, the application can automatically reproduce a high-definition image through the preset image generation model, reduces the time consumption of manual operation, improves the efficiency of generating a model clothing material picture, and thus can meet the operation requirements of the e-commerce industry.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image generation method, apparatus, device, and storage medium. Background Technology

[0002] Currently, in the e-commerce product image display scenario for clothing, in order to obtain high-definition model display images that match the characteristics of the product, it is generally necessary to first have a professional photography team take high-definition pictures of models wearing clothing, then have designers manually edit the images to ensure that the image quality matches the characteristics of the product, and finally upload the processed images to the e-commerce platform for product display.

[0003] However, the aforementioned image resources rely on manual processing, which is not only costly but also inefficient, making it difficult to meet the demands of rapid iteration and efficient operation in the e-commerce industry. For example, for an apparel e-commerce company, this process requires hiring models, photographers, and post-production designers, spending half a day or even longer to complete the shooting and online launch of a set of clothing. During promotional seasons or when new products are launched rapidly, it is generally difficult to provide a sufficient number of high-quality e-commerce images in a timely manner, thereby affecting the sales performance and user experience of the products. Summary of the Invention

[0004] The main objective of this application is to provide an image generation method, apparatus, device, and storage medium, which aims to solve the technical problem that traditional image generation methods are inefficient and cannot meet the operational needs of goods.

[0005] To achieve the above objectives, this application proposes an image generation method, the method comprising:

[0006] Get the current clothing image and the current model image; The current clothing image and the current model image are input into a preset image generation model, which is constructed based on a diffusion model and trained by wavelet transform. The clothing features of the current clothing image are extracted using the preset image generation model, and the clothing features are fused with the current model image to obtain a model clothing material image.

[0007] In one embodiment, the step of extracting clothing features from the current clothing image using the preset image generation model and fusing the clothing features with the current model image to obtain a model clothing material image includes: The preset image generation model includes a clothing encoder and a fusion generation network; The clothing encoder extracts features from the current clothing image to obtain the clothing features corresponding to the current clothing image. The clothing features and the current model image are input into the fusion generation network for fusion to obtain a model clothing material image.

[0008] In one embodiment, the preset image generation model further includes a model image encoder; the step of inputting the clothing features and the current model image into the fusion generation network for fusion to obtain a model clothing material image includes: The model features are obtained by extracting features from the current model image using the model image encoder. The current model image is adjusted based on the model's characteristics to obtain an adjusted model image. The adjustment operation includes posture adjustment or body shape adjustment. The adjusted model image and the clothing features are input into the fusion generation network for fusion to obtain the model clothing material image.

[0009] In one embodiment, the preset image generation model further includes a text encoder, and the step of inputting the adjusted model image and the clothing features into the fusion generation network for fusion to obtain the model clothing material image includes: Get the text suggestions entered by the user; The text encoder extracts features from the text prompts to obtain semantic features; Based on the semantic features, the adjusted model image and the clothing features are fused through the fusion generation network to obtain a model clothing material image.

[0010] In one embodiment, prior to the steps of acquiring the current clothing image and the current model image, the method further includes: Obtain multiple scene model images containing clothing and models, and corresponding sample clothing images; Determine the proportion of human body in each of the scene model images, and remove images whose human body proportion is lower than a preset proportion threshold from each of the scene model images; Extract sample model images from the removed scene model images; The preset diffusion model is trained based on the sample model image and the sample clothing image, and the model parameters are fine-tuned by wavelet transform based on the training results to obtain the preset image generation model.

[0011] In one embodiment, the step of extracting sample model images from the removed scene model images includes: The model image after removal is subjected to portrait extraction to obtain the model portrait region, and the ROI mask of the model portrait region is determined; The variance of the ROI mask is obtained by performing variance processing on the Laplacian operator; If the variance of the entire image reaches a preset variance threshold, the scene model image corresponding to the preset variance threshold will be used as the sample model image.

[0012] In one embodiment, the step of training a preset diffusion model based on the sample model image and the sample clothing image, and fine-tuning the model parameters through wavelet transform based on the training results to obtain a preset image generation model includes: The preset diffusion model is trained based on the sample model images and the sample clothing images to obtain the training results; If the accuracy of the training result does not reach a preset threshold, the high-frequency features between the sample model image and the sample clothing image are decomposed by wavelet transform to obtain high-frequency components, and the spatial loss of the model is determined based on the high-frequency components. The model hyperparameters are fine-tuned based on the spatial loss to obtain a preset image generation model.

[0013] Furthermore, to achieve the above objectives, this application also proposes an image generation apparatus, the apparatus comprising: The image acquisition module is used to acquire the current clothing image and the current model image; An image input module is used to input the current clothing image and the current model image into a preset image generation model, which is constructed based on a diffusion model and trained by wavelet transform. The feature fusion module is used to extract clothing features from the current clothing image through the preset image generation model, and to fuse the clothing features with the current model image to obtain a model clothing material image.

[0014] In addition, to achieve the above objectives, this application also proposes an image generation apparatus, the apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image generation method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the image generation method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: The image generation method of this application includes: acquiring a current clothing image and a current model image; inputting the current clothing image and the current model image into a preset image generation model, wherein the preset image generation model is constructed based on a diffusion model and trained by wavelet transform; extracting clothing features from the current clothing image through the preset image generation model, and fusing the clothing features with the current model image to obtain a model clothing material image.

[0017] This application first acquires the current clothing image and the current model image, then inputs them into a preset image generation model built on a diffusion model and trained using wavelet transform. The model then extracts clothing features and fuses these features with the current model image to obtain the model clothing source image. Compared to existing processes that rely on photography teams and manual image editing, this application can automatically reproduce high-definition images using a preset image generation model, reducing the time spent on manual operations and improving the efficiency of generating model clothing source images, thus meeting the operational needs of the e-commerce industry. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the image generation method of this application. Figure 2 This is an overall architecture diagram of the image generation process provided in Embodiment 1 of this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the image generation method of this application; Figure 4 This is a block diagram of the module structure of the image generation apparatus according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware operating environment involved in the image generation device in the embodiments of this application.

[0021] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] The main solution proposed in this application is as follows: Currently, in the e-commerce material image display scenario for clothing products, in order to obtain high-definition model display images that match the characteristics of the products, it is generally necessary to first have a professional photography team take high-definition pictures of models wearing clothing, then have designers manually edit the images to ensure that the image quality matches the characteristics of the products, and finally upload the processed images to the e-commerce platform for product display.

[0024] However, the aforementioned image resources rely on manual processing, which is not only costly but also inefficient, making it difficult to meet the demands of rapid iteration and efficient operation in the e-commerce industry. For example, for an apparel e-commerce company, this process requires hiring models, photographers, and post-production designers, spending half a day or even longer to complete the shooting and online launch of a set of clothing. During promotional seasons or when new products are launched rapidly, it is generally difficult to provide a sufficient number of high-quality e-commerce images in a timely manner, thereby affecting the sales performance and user experience of the products.

[0025] To address the aforementioned issues, this application first acquires the current clothing image and the current model image, then inputs them into a preset image generation model built on a diffusion model and trained using wavelet transform. The model then extracts clothing features and fuses these features with the current model image to obtain a model clothing source image. Compared to existing processes that rely on photography teams and manual image editing, this application can automatically replicate high-definition images using the preset image generation model, reducing the time consumed by manual operations and improving the efficiency of generating model clothing source images, thereby meeting the operational needs of the e-commerce industry.

[0026] It should be noted that the executing entity of this application embodiment can be a computing service device with data processing, model training, and image generation functions, such as a mobile phone, tablet computer, or personal computer, or an electronic device capable of realizing the above functions, an image generation device executing the image generation method of this application, etc. This embodiment does not limit it. The following uses an image generation device (hereinafter referred to as the device) as an example to describe this embodiment and the following embodiments.

[0027] Based on this, this application proposes an image generation method according to a first embodiment, referring to... Figure 1 , Figure 1This is a flowchart illustrating an embodiment of the image generation method of this application. In this embodiment, the image generation method may include steps S10 to S30: Step S10: Obtain the current clothing image and the current model image.

[0028] It should be noted that the current clothing image can be a flat image of the clothing to be displayed in the clothing e-commerce scenario or a product image that is not worn by a model (such as a 2.5D clothing image).

[0029] It should also be noted that the current model image can be a low-resolution image or a model image from a non-target scene (such as a non-white background or a non-target pose). For example, an e-commerce company may already have a low-resolution image of a model wearing clothing taken with a mobile phone, but the material requires a high-resolution image, and the low-resolution image does not meet the requirements for product display. Therefore, the image generation method of this embodiment can be used in this case.

[0030] In practical use, in apparel e-commerce scenarios, merchants already have product images (such as 2.5D clothing images) of the clothing to be displayed. During promotional seasons or when new products are rapidly launched, merchants can simply take a low-resolution photo of a model wearing the clothing (i.e., the current model image) with their mobile phone. The device can then acquire the aforementioned current model image and the corresponding current clothing image for subsequent image processing.

[0031] Step S20: Input the current clothing image and the current model image into a preset image generation model. The preset image generation model is constructed based on a diffusion model and trained by wavelet transform.

[0032] Understandably, the preset image generation model can be an AI model built on a diffusion model and optimized and trained using wavelet transform. The diffusion model is a probability-based generative model whose core is to gradually add noise to the data (i.e., the image) to make it random noise, and then learn to gradually remove noise from the noise and reconstruct the original data through a reverse process.

[0033] For example, during model training, the model first simulates noise degradation on a low-resolution input image, and then learns how to remove noise and enhance details (such as texture and edges) through training, ultimately generating an output image that is consistent with the distribution of real high-resolution images. Its advantage is that it can generate high-quality and diverse results and the training is more stable.

[0034] Step S30: Extract clothing features from the current clothing image using the preset image generation model, and fuse the clothing features with the current model image to obtain a model clothing material image.

[0035] It should be noted that clothing features can be visual information in the current clothing image that can characterize its key attributes such as style, color, texture, and structure. For example, the shape of the skirt of a dress, the texture of the fabric folds, the color distribution of the pattern, or the spatial layout of the clothing in the image. This embodiment does not limit this. The model clothing material image can be a high-definition, standardized display image generated by fusing clothing features (such as style, color, and texture) with the model's image (such as posture, body shape, and background) in an e-commerce scenario.

[0036] In practical use, after receiving the current clothing image and the current model image, the preset image generation model can first extract clothing features from the current clothing image, and then combine the clothing features (such as style and color) with the current model image in the latent space to generate a high-definition model clothing material image. For example, when a flat image of a dress and a low-resolution image of the model are input, the preset image generation model can automatically extract the drape and pattern details of the dress, combine it with the model's posture and a white background, and generate a high-definition model image of the dress that meets the requirements for e-commerce listing. This preserves the realistic texture of the clothing and improves the efficiency of consumers' purchasing decisions through standardized presentation, avoiding the inefficiency and errors of traditional image editing that relies on manually drawing details.

[0037] Furthermore, in order to obtain the aforementioned model clothing source images, refer to Figure 2 , Figure 2 This is an overall architecture diagram of the image generation process provided in Embodiment 1 of this application. Figure 2 As shown, in this embodiment, the step of extracting clothing features from the current clothing image using the preset image generation model and fusing the clothing features with the current model image to obtain a model clothing material image includes: The preset image generation model includes a clothing encoder and a fusion generation network; The clothing encoder extracts features from the current clothing image to obtain the clothing features corresponding to the current clothing image. The clothing features and the current model image are input into the fusion generation network for fusion to obtain a model clothing material image.

[0038] It should be noted that the clothing encoder can be a module in a pre-defined image generation model responsible for parsing the core features of a clothing image. This clothing encoder can employ multi-scale convolution and attention mechanisms, and this embodiment does not impose any restrictions on this. For example, after inputting a flat image of a dress, the clothing encoder can ignore background interference and accurately extract clothing features such as skirt folds and neckline design, providing a foundation for subsequent fusion.

[0039] It should also be noted that the fusion generative network can be a deep learning module that dynamically fuses clothing features with model images in the latent space.

[0040] In this embodiment, the current clothing image is first processed by a clothing encoder to extract features, thus obtaining clothing features. Then, the clothing features and the current model image are input into a fusion generation network for fusion to obtain a model clothing source image. In this way, the clothing encoder can accurately extract clothing features, and the fusion generation network can specifically optimize the current model image with clothing features, avoiding the color deviation problem of manual image editing and further improving the clarity of the clothing in the model clothing source image.

[0041] Furthermore, in order to adjust the current model image, such as Figure 2 As shown, in this embodiment, the preset image generation model further includes a model image encoder; the step of inputting the clothing features and the current model image into the fusion generation network for fusion to obtain the model clothing material image includes: The model features are obtained by extracting features from the current model image using the model image encoder. The current model image is adjusted based on the model's characteristics to obtain an adjusted model image. The adjustment operation includes posture adjustment or body shape adjustment. The adjusted model image and the clothing features are input into the fusion generation network for fusion to obtain the model clothing material image.

[0042] Understandably, the model image encoder can be a module in a pre-defined image generation model that parses the structured features of the model image, such as posture and shape. For example, after inputting a low-resolution standing image of a model, the model image encoder extracts its skeletal key points (such as shoulder width and waistline ratio) and posture angles (such as a 30-degree side profile) through multi-scale convolution, generating standardized feature vectors (i.e., model features), providing a basis for subsequent adjustments to the model image.

[0043] It is also understandable that the adjustment operation can be a dynamic optimization operation based on the model's features on the original current model image, including posture correction (such as converting a side view to a front view) and body standardization (such as standardizing the model's waist-to-hip ratio). For example, the input current model image may have a slightly hunched posture, and the model can adjust it to a standard posture with the chest out and head up based on the model's features to maintain the fit of the clothing.

[0044] This embodiment analyzes the model's features in the current model image using a model image encoder, and combines these features to dynamically and quickly adjust the pose or shape. Compared to traditional methods that require post-processing of the image, this significantly improves efficiency.

[0045] Furthermore, in order to generate model clothing source images as required, such as Figure 2As shown, in this embodiment, the preset image generation model is further equipped with a text encoder. The step of inputting the adjusted model image and the clothing features into the fusion generation network for fusion to obtain the model clothing material image includes: Get the text suggestions entered by the user; The text encoder extracts features from the text prompts to obtain semantic features; Based on the semantic features, the adjusted model image and the clothing features are fused through the fusion generation network to obtain a model clothing material image.

[0046] It should be noted that the text prompts can be natural language descriptions entered by the user, used to specify the generation requirements for the model's clothing image. For example, the user can enter "generate a high-resolution image of a model wearing a red dress on the beach, with the skirt flowing in the wind." The device will then parse the text to obtain key instructions such as style, scene, and movement, and make adjustments during the fusion process.

[0047] It should also be noted that the text encoder can be a module in a pre-defined image generation model responsible for converting natural language into structured semantic features. For example, given the input "red dress + beach background", the text encoder extracts features such as color (red), clothing type (dress), and scene (beach) to generate semantic features that can be understood by the fused generative network.

[0048] This embodiment uses a text encoder to parse user-inputted text prompts, obtaining semantic features. These features are then combined with adjusted model images and clothing features for customized fusion. This enables rapid customization of model and clothing source images, resulting in standardized source images that align with the brand's tone, without relying on designer experience. Previously, the process from shooting and editing product images to product launch could take half a day; now, using raw model images, this process can be completed in just tens of seconds, significantly improving the efficiency of new model image uploads.

[0049] This application provides an image generation method. First, it acquires an image of the current clothing and an image of the current model, then inputs them into a preset image generation model built on a diffusion model and trained using wavelet transform. Next, the model extracts clothing features and fuses these features with the current model image to obtain a model clothing source image. Compared to existing methods that rely on photography teams and manual image editing, this embodiment can automatically replicate high-definition images using the preset image generation model, reducing the time consumed by manual operations and improving the efficiency of generating model clothing source images, thereby meeting the operational needs of the e-commerce industry.

[0050] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the above embodiment can be referred to the above description, and will not be repeated hereafter. On this basis, a second embodiment of the dialogue method of this application is proposed, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of the image generation method of this application. To obtain the aforementioned preset image generation model, as shown... Figure 3 As shown, in this embodiment, before the steps of obtaining the current clothing image and the current model image, the method further includes: Step S01: Obtain multiple scene model images containing clothing and models, and corresponding sample clothing images.

[0051] It should be noted that the scene model images can be diverse pictures of models wearing clothing taken in e-commerce scenarios, covering different backgrounds (such as photography studios, outdoors), poses (such as standing, walking), and lighting conditions. For example, a clothing brand might take 100 pictures of models wearing the same dress in scenes such as beaches, forests, and city streets, to train subsequent models.

[0052] It should also be noted that the sample clothing images can be flat images corresponding to the clothing in the scene model images or non-model wearing images, used to provide the original details of the clothing (such as texture, color, and pattern).

[0053] Step S02: Determine the proportion of human body in each of the scene model images, and remove the images corresponding to human body proportions lower than a preset proportion threshold from each of the scene model images.

[0054] Understandably, the human body proportion can be the area ratio of the model in the scene model image. The preset proportion threshold can be a critical value for filtering scene model images (such as 30%, 40%, etc., which is not limited in this embodiment) to ensure that the model's main body is clear in the remaining images, which is conducive to the model learning the integration relationship between clothing and human body.

[0055] For example, in a 1000×1000 pixel image, if the model area is 400×600 pixels, then the human body accounts for 24%. In this case, data where the human body accounts for less than 40% of the image can be removed to filter effective training data and avoid the model learning bias caused by an excessively high background ratio.

[0056] Step S03: Extract sample model images from the removed scene model images.

[0057] The sample model images can be clear images extracted from the selected scene model images, serving as the core training set for model learning.

[0058] Step S04: Train the preset diffusion model based on the sample model image and the sample clothing image, and fine-tune the model parameters based on the training results using wavelet transform to obtain the preset image generation model.

[0059] This embodiment first acquires multiple scene model images containing clothing and models, along with corresponding sample clothing images. Then, based on the proportion of the human body in the scene model images, images with a human body proportion lower than a preset threshold are removed from each scene model image. Next, sample model images are extracted from the removed scene model images as the core training set for model learning. Finally, a preset diffusion model is trained using the sample model images and sample clothing images, and the model parameters are fine-tuned using wavelet transform based on the training results to obtain a preset image generation model. The final preset image generation model can significantly reduce the cost of producing e-commerce material images and meet the operational needs of rapid new product launches.

[0060] Furthermore, in order to obtain the aforementioned sample model images, in this embodiment, the step of extracting sample model images from the removed scene model images includes: The model image after removal is subjected to portrait extraction to obtain the model portrait region, and the ROI mask of the model portrait region is determined; The variance of the ROI mask is obtained by performing variance processing on the Laplacian operator; If the variance of the entire image reaches a preset variance threshold, the scene model image corresponding to the preset variance threshold will be used as the sample model image.

[0061] It should be noted that the model's portrait region can be the image area containing the model's subject after portrait extraction, usually represented by pixel coordinate ranges or a binary mask. The ROI mask (Region of Interest Mask) can be a binary image that marks the model's portrait region, where the pixel value of the model is 1 (foreground) and the pixel value of the background is 0 (background).

[0062] It's also understandable that the Laplacian operator can be a second-order differential operator used for image edge detection. It highlights high-frequency information (such as edges and textures) in an image by calculating the gray-level differences in the neighborhood of a pixel. For example, after applying the Laplacian operator to a ROI mask, the model's contours (such as hair edges and clothing folds) will show high response values, while smooth areas will have lower response values, thus quantifying the richness of image detail. The overall image variance can be a statistical measure calculated based on the Laplacian operator processing results, reflecting the dispersion of pixel values ​​in the ROI mask. A higher variance value indicates more complex image edges or textures.

[0063] It should also be noted that the preset variance threshold can be a critical value for filtering sample model images, used to determine whether the image contains enough detail to support model training. For example, if the threshold is set to 600, images with a variance ≥ 600 (such as models wearing printed dresses or making exaggerated poses) will be retained, while images with a variance < 600 (such as models wearing solid-color T-shirts or standing upright) may be discarded, ensuring that the training data contains diverse clothing features.

[0064] Because this embodiment focuses on the model subject through portrait extraction, combines the Laplacian operator to quantify image details, and finally selects high-value training samples based on the variance threshold, it significantly improves the training efficiency and generation quality of the image generation model.

[0065] Furthermore, to improve the accuracy of the model, in this embodiment, the step of training a preset diffusion model based on the sample model image and the sample clothing image, and fine-tuning the model parameters through wavelet transform based on the training results to obtain a preset image generation model includes: The preset diffusion model is trained based on the sample model images and the sample clothing images to obtain the training results; If the accuracy of the training result does not reach a preset threshold, the high-frequency features between the sample model image and the sample clothing image are decomposed by wavelet transform to obtain high-frequency components, and the spatial loss of the model is determined based on the high-frequency components. The model hyperparameters are fine-tuned based on the spatial loss to obtain a preset image generation model.

[0066] It should be noted that the training results can be the output of the diffusion model after training on sample model images and sample clothing images, and usually include indicators such as sharpness and color reproduction. The preset threshold can be a critical value for judging whether the training results meet the standard. For example, if the preset threshold is that the SSIM (structural similarity) between the generated image and the real image is ≥0.85, then the model meets the standard.

[0067] Understandably, wavelet transform is a multi-scale analysis tool that optimizes the quality of an image by decomposing its high-frequency (detailed texture) and low-frequency (structural contour) features. High-frequency components can be the parts of the image containing detailed information after wavelet transform decomposition, such as clothing texture, model facial expressions, and background lighting variations. For example, applying wavelet transform to sample model images and sample clothing images can extract high-frequency components such as clothing folds and model hair strands, which can be used to analyze the model's shortcomings in detail reproduction.

[0068] It is also understandable that spatial loss can be a model optimization metric based on high-frequency components, reflecting the difference in detail between the generated image and the real image. Using wavelet latent spatial enhancement methods, it is possible to preserve low-frequency information while emphasizing high-frequency components, thereby significantly enhancing the fine details and rich textures in high-resolution image generation.

[0069] Specifically, to facilitate understanding of the fine-tuning process of wavelet transform, an example is given below. Consider the diffusion process of the diffusion model, namely: ; in, This represents a sample in the distribution of the original data (i.e., sample model images and sample clothing images). This represents noise in a standard normal distribution. and These are hyperparameters in the diffusion model formula, which control the weights of data distribution and noise, respectively. This represents the potential representation of the diffusion model at time step t.

[0070] At this point, the parameterization function of the original training objective in the latent diffusion model, used for noise prediction, can be defined as follows: ; in, The parameters representing a neural network (such as convolutional UNet or DiT). This represents the actual noise added to the data at time step t.

[0071] Then, the model can use a corrected flow to predict the flow caused by… The speed of parameterization makes the predicted noise or data closer to the true value. Its objective function is as follows: ; in, It is the speed parameterized by the neural network, used for predicting noise adjustment or data reconstruction.

[0072] Finally, wavelet transform can be used to decompose the low-frequency approximation and high-frequency details of the latent features to measure the difference between the model output and the target. The training objective of the wavelet loss function is defined as follows: ; in, The expected value represents the average of all possible samples or data distributions; The loss weights, which may vary with time step t, are used to adjust the loss contribution at different time steps; f() represents the discrete wavelet transform, used to decompose the signal into components of different frequencies. Therefore, the formula simultaneously integrates low-frequency information and high-frequency details, contributing to comprehensive optimization of high-definition image generation. It also supports multiple diffusion models, enabling seamless integration with traditional noise prediction methods.

[0073] Furthermore, to further improve the model, in addition to fine-tuning with wavelet transform, potential optimizations can be made through Attention Diversification Loss (ADL) and TV-LPIPS Improved Perceptual Loss. Specifically, TV-LPIPS Improved Perceptual Loss aims to reduce variations between adjacent pixels in flat regions to suppress high-frequency artifacts while preserving sharp edges. Attention Diversification Loss, used at the feature level to address periodic imperfections, aims to reduce similarity between samples and enhance the diversity of attention. This significantly enhances fine details and rich textures in the generation of high-definition model clothing source images.

[0074] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image generation method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0075] This application also provides an image generation apparatus, please refer to... Figure 4 , Figure 4 This is a block diagram of the module structure of the image generation device according to an embodiment of this application; in this embodiment, the image generation device includes: Image acquisition module 401 is used to acquire the current clothing image and the current model image; Image input module 402 is used to input the current clothing image and the current model image into a preset image generation model, wherein the preset image generation model is constructed based on a diffusion model and trained by wavelet transform. The feature fusion module 403 is used to extract the clothing features of the current clothing image through the preset image generation model, and to fuse the clothing features with the current model image to obtain a model clothing material image.

[0076] In one implementation, the preset image generation model includes a clothing encoder and a fusion generation network; the feature fusion module 403 is further used to extract features from the current clothing image through the clothing encoder to obtain clothing features corresponding to the current clothing image; and to input the clothing features and the current model image into the fusion generation network for fusion to obtain a model clothing material image.

[0077] As one implementation, the preset image generation model is further provided with a model image encoder; the feature fusion module 403 is further used to extract features from the current model image through the model image encoder to obtain model features; to perform adjustment operations on the current model image according to the model features to obtain an adjusted model image, the adjustment operations including posture adjustment or body shape adjustment; and to input the adjusted model image and the clothing features into the fusion generation network for fusion to obtain a model clothing material image.

[0078] As one implementation, the preset image generation model is further equipped with a text encoder, and the feature fusion module 403 is further used to obtain text prompts input by the user; extract features from the text prompts through the text encoder to obtain semantic features; and fuse the adjusted model image and the clothing features through the fusion generation network according to the semantic features to obtain a model clothing material image.

[0079] In one implementation, the image generation device further includes a model training module, used to acquire multiple scene model images containing clothing and models and corresponding sample clothing images; determine the proportion of human body in each scene model image, and remove images whose human body proportion is lower than a preset proportion threshold from each scene model image; extract sample model images from the removed scene model images; train a preset diffusion model based on the sample model images and the sample clothing images, and fine-tune the model parameters based on the training results using wavelet transform to obtain a preset image generation model.

[0080] As one implementation, the model training module is further used to extract the human image from the removed scene model image to obtain the model human image region and determine the ROI mask of the model human image region; to process the variance of the ROI mask by the Laplacian operator to obtain the full image variance; and when the full image variance reaches a preset variance threshold, the scene model image corresponding to the preset variance threshold is used as the sample model image.

[0081] In one implementation, the model training module is further configured to train a preset diffusion model based on the sample model image and the sample clothing image to obtain a training result; if the accuracy of the training result does not reach a preset threshold, the high-frequency features between the sample model image and the sample clothing image are decomposed by wavelet transform to obtain high-frequency components, and the spatial loss of the model is determined based on the high-frequency components; the model hyperparameters are fine-tuned based on the spatial loss to obtain a preset image generation model.

[0082] Other embodiments or specific implementations of the image generation apparatus of this application can be found in the above-described method embodiments, and will not be repeated here.

[0083] The image generation apparatus provided in this application, employing the image generation method described in the above embodiments, can solve the technical problem that traditional material image generation methods are inefficient and unable to meet the operational needs of goods. Compared with the prior art, the beneficial effects of the image generation apparatus provided in this application are the same as those of the image generation method described in the above embodiments, and other technical features in the image generation apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0084] This application provides an image generation apparatus, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image generation methods described in the above embodiments.

[0085] The following is for reference. Figure 5 , Figure 5 This is a schematic diagram of the hardware operating environment involved in the image generation device in the embodiments of this application, showing a structural schematic diagram suitable for implementing the image generation device in the embodiments of this application. The image generation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The image generation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0086] like Figure 5As shown, the image generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the image generation device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the image generating device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows image generating devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0087] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0088] The image generation device provided in this application, employing the image generation method described in the above embodiments, can solve the technical problem that traditional material image generation methods are inefficient and unable to meet the operational needs of goods. Compared with the prior art, the beneficial effects of the image generation device provided in this application are the same as those of the image generation method provided in the above embodiments, and other technical features of this image generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0089] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0091] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the image generation method described in the above embodiments.

[0092] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0093] The aforementioned computer-readable storage medium may be included in the image generating apparatus or may exist independently without being assembled into the image generating apparatus.

[0094] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an image generation device, cause the image generation device to: acquire a current clothing image and a current model image; input the current clothing image and the current model image into a preset image generation model, the preset image generation model being constructed based on a diffusion model and trained using wavelet transform; extract clothing features from the current clothing image using the preset image generation model, and fuse the clothing features with the current model image to obtain a model clothing material image.

[0095] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0098] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image generation method. This solves the technical problem that traditional image generation methods are inefficient and cannot meet the operational needs of goods. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image generation method provided in the above embodiments, and will not be repeated here.

[0099] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. An image generation method, characterized in that, The method includes: Get the current clothing image and the current model image; The current clothing image and the current model image are input into a preset image generation model, which is constructed based on a diffusion model and trained by wavelet transform. The clothing features of the current clothing image are extracted using the preset image generation model, and the clothing features are fused with the current model image to obtain a model clothing material image.

2. The method as described in claim 1, characterized in that, The step of extracting clothing features from the current clothing image using the preset image generation model and fusing the clothing features with the current model image to obtain a model clothing material image includes: The preset image generation model includes a clothing encoder and a fusion generation network; The clothing encoder extracts features from the current clothing image to obtain the clothing features corresponding to the current clothing image. The clothing features and the current model image are input into the fusion generation network for fusion to obtain a model clothing material image.

3. The method as described in claim 2, characterized in that, The preset image generation model also includes a model image encoder; the step of inputting the clothing features and the current model image into the fusion generation network for fusion to obtain the model clothing material image includes: The model features are obtained by extracting features from the current model image using the model image encoder. The current model image is adjusted based on the model's characteristics to obtain an adjusted model image. The adjustment operation includes posture adjustment or body shape adjustment. The adjusted model image and the clothing features are input into the fusion generation network for fusion to obtain the model clothing material image.

4. The method as described in claim 3, characterized in that, The preset image generation model also includes a text encoder. The step of inputting the adjusted model image and the clothing features into the fusion generation network for fusion to obtain the model clothing material image includes: Get the text suggestions entered by the user; The text encoder extracts features from the text prompts to obtain semantic features; Based on the semantic features, the adjusted model image and the clothing features are fused through the fusion generation network to obtain a model clothing material image.

5. The method according to any one of claims 1 to 4, characterized in that, Before the steps of acquiring the current clothing image and the current model image, the method further includes: Obtain multiple scene model images containing clothing and models, and corresponding sample clothing images; Determine the proportion of human body in each of the scene model images, and remove images whose human body proportion is lower than a preset proportion threshold from each of the scene model images; Extract sample model images from the removed scene model images; The preset diffusion model is trained based on the sample model image and the sample clothing image, and the model parameters are fine-tuned by wavelet transform based on the training results to obtain the preset image generation model.

6. The method as described in claim 5, characterized in that, The step of extracting sample model images from the removed scene model images includes: The model image after removal is subjected to portrait extraction to obtain the model portrait region, and the ROI mask of the model portrait region is determined; The variance of the ROI mask is obtained by performing variance processing on the Laplacian operator; If the variance of the entire image reaches a preset variance threshold, the scene model image corresponding to the preset variance threshold will be used as the sample model image.

7. The method as described in claim 5, characterized in that, The step of training a preset diffusion model based on the sample model image and the sample clothing image, and fine-tuning the model parameters using wavelet transform based on the training results to obtain a preset image generation model includes: The preset diffusion model is trained based on the sample model images and the sample clothing images to obtain the training results; If the accuracy of the training result does not reach a preset threshold, the high-frequency features between the sample model image and the sample clothing image are decomposed by wavelet transform to obtain high-frequency components, and the spatial loss of the model is determined based on the high-frequency components. The model hyperparameters are fine-tuned based on the spatial loss to obtain a preset image generation model.

8. An image generation apparatus, characterized in that, The device includes: The image acquisition module is used to acquire the current clothing image and the current model image; An image input module is used to input the current clothing image and the current model image into a preset image generation model, which is constructed based on a diffusion model and trained by wavelet transform. The feature fusion module is used to extract clothing features from the current clothing image through the preset image generation model, and to fuse the clothing features with the current model image to obtain a model clothing material image.

9. An image generation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image generation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the image generation method as described in any one of claims 1 to 7.

Images