Image generation method and device, electronic equipment and storage medium

Abstract

The invention discloses an image generation method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a target commodity map, and converting the target commodity map into a grey-scale map and a masking map; obtaining a target background image, and extracting low-frequency feature mapping representing light information from the target background image; generating a diffusion model through the image, and generating a commodity display graph according to the grey-scale map, the masking map and the low-frequency feature mapping. The illumination information of the background image and the commodity image are utilized to generate the required commodity display image, so that fusion of the commodity image and the background image in the commodity display image is more natural, and the quality of the commodity display image is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image generation method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of e-commerce, product display pictures play a vital role in attracting consumers' attention and improving purchase conversion rates.

[0003] At present, the production process of product display pictures is as follows: first generate a background picture separately, and then directly place the product picture on the background picture to use it as the product display picture. However, the product display pictures produced in this way have problems of unnatural and uncoordinated visual effects, that is, the quality of product display pictures produced by the existing method is low. Therefore, how to generate high-quality product display pictures has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The invention provides an image generating method, an apparatus, an electronic device and a storage medium.

[0005] According to another aspect of the present invention, there is provided an image generation method, comprising:

[0006] Obtain a target product image, and convert the target product image into a grayscale image and a mask image;

[0007] Obtain a target background image, and extract a low-frequency feature map representing light information from the target background image;

[0008] The diffusion model is generated through images, and the product display image is generated according to the grayscale image, mask image, and low-frequency feature map.

[0009] In some embodiments, extracting a low-frequency feature map representing light information from a target background image includes:

[0010] A pre-trained light information extraction model is used to extract a low-frequency feature map representing light information from the target background image.

[0011] Among them, the light information extraction model includes multiple light extractors connected in series; the structure of any light extractor includes a first encoding layer, an intermediate layer, a first decoding layer and a second decoding layer composed of a convolutional network.

[0012] In some embodiments, the process of training the light information extraction model includes:

[0013] Constructing training samples; wherein each training sample includes a high-resolution sample image and a low-resolution sample image, and the image content of the high-resolution sample image and the low-resolution sample image is the same;

[0014] For any training sample, noise is added to the low-resolution sample image in the training sample to obtain a noise image;

[0015] The noisy image is input into the light information extraction model for denoising and image generation processing to obtain a target image with the same resolution as the high-resolution sample image;

[0016] The loss is calculated based on the high-resolution sample image and the target image, and the parameters of the light information extraction model are adjusted according to the loss calculation results.

[0017] In some embodiments, the method further comprises:

[0018] Extract grayscale information from grayscale images through auxiliary image generation model;

[0019] Low-frequency feature maps are processed through the cross-attention mechanism.

[0020] In some embodiments, a diffusion model is generated through an image, and a product display image is generated according to a grayscale image, a mask image, and a low-frequency feature map, including:

[0021] The grayscale information extracted from the grayscale image and the processed low-frequency feature map are used as image generation conditions and input into the image generation diffusion model;

[0022] The image generation diffusion model generates a product display image based on the input mask image according to the image generation conditions.

[0023] In some embodiments, the method further comprises:

[0024] Obtain the original product display image, and perform foreground separation processing on the original product display image through a semantic segmentation model to obtain an original foreground image and an original background image;

[0025] The original foreground image is used as the target product image, and the original background image is used as the target background image.

[0026] According to another aspect of the present invention, there is provided an image generating device, comprising:

[0027] The first processing module is used to obtain a target product image and convert the target product image into a grayscale image and a mask image;

[0028] The second processing module is used to obtain a target background image and extract a low-frequency feature map representing light information from the target background image;

[0029] The image generation module is used to generate a diffusion model through an image and generate a product display image based on a grayscale image, a mask image, and a low-frequency feature map.

[0030] In some embodiments, the second processing module is specifically configured to:

[0031] A pre-trained light information extraction model is used to extract a low-frequency feature map representing light information from the target background image.

[0032] Among them, the light information extraction model includes multiple light extractors connected in series; the structure of any light extractor includes a first encoding layer, an intermediate layer, a first decoding layer and a second decoding layer composed of a convolutional network.

[0033] In some embodiments, the apparatus further includes a model training module for training the light information extraction model, and the model training module is specifically used to:

[0034] Constructing training samples; wherein each training sample includes a high-resolution sample image and a low-resolution sample image, and the image content of the high-resolution sample image and the low-resolution sample image is the same;

[0035] For any training sample, noise is added to the low-resolution sample image in the training sample to obtain a noise image;

[0036] The noisy image is input into the light information extraction model for denoising and image generation processing to obtain a target image with the same resolution as the high-resolution sample image;

[0037] The loss is calculated based on the high-resolution sample image and the target image, and the parameters of the light information extraction model are adjusted according to the loss calculation results.

[0038] In some embodiments, the apparatus further comprises:

[0039] A grayscale extraction module is used to extract grayscale information from a grayscale image by using an auxiliary image generation model;

[0040] The feature processing module is used to process low-frequency feature maps through a cross-attention mechanism.

[0041] In some embodiments, the image generation module is specifically used to:

[0042] The grayscale information extracted from the grayscale image and the processed low-frequency feature map are used as image generation conditions and input into the image generation diffusion model;

[0043] The image generation diffusion model generates a product display image based on the input mask image according to the image generation conditions.

[0044] In some embodiments, the apparatus further comprises:

[0045] A foreground separation module is used to obtain the original product display image and perform foreground separation processing on the original product display image through a semantic segmentation model to obtain an original foreground image and an original background image;

[0046] The determination module is used to use the original foreground image as the target product image and the original background image as the target background image.

[0047] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0048] at least one processor; and

[0049] a memory communicatively connected to at least one processor; wherein,

[0050] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the image generating method according to the embodiment of the present invention.

[0051] According to another aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions. The computer instructions are used to enable a processor to implement the image generation method according to an embodiment of the present invention when executed.

[0052] The technical solution of the embodiment of the present invention utilizes the lighting information of the background image and the product image to generate the required product display image, so that the fusion of the product image and the background image in the product display image is more natural, thereby significantly improving the quality of the product display image.

[0053] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1a is a flow chart of an image generating method provided by an embodiment of the present invention;

[0056] Figure 1b is a schematic structural diagram of a single light extractor provided by an embodiment of the present invention;

[0057] Figure 1c It is an overall flow chart of extracting low-frequency feature maps through a light information extraction model of six light extractors provided by an embodiment of the present invention;

[0058] Figure 1dIt is a specific flow chart of obtaining low-frequency feature mapping through 6 light extractors provided by an embodiment of the present invention;

[0059] Figure 2 is a flow chart of another image generating method provided by an embodiment of the present invention;

[0060] Figure 3a is a flow chart of another image generating method provided by an embodiment of the present invention;

[0061] Figure 3b The present invention provides a flowchart of a noise image generating a target image through a light information extraction model;

[0062] Figure 4 is a structural schematic diagram of an image generating device provided by an embodiment of the present invention;

[0063] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing the image generating method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0065] In the embodiments of the present invention, the applicant found that when directly inserting a product image into a pre-made background image to generate a product display image, the quality of the generated product display image is poor (for example, it is not natural and coordinated in visual display) because the lighting relationship between the background and the product image is not considered during the generation of the product display image. Based on this, the present invention proposes an image generation method for generating a high-quality product display image by referring to the light information in the background image. The specific process of the method can be seen in the following embodiments.

[0066] Embodiment 1

[0067] Figure 1a A flowchart of an image generation method is provided for an embodiment of the present invention. This embodiment can be applicable to scenarios of generating high-quality product display images. The method can be executed by an image generation device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device.

[0068] like Figure 1a As shown, the image generation method includes:

[0069] S101: Obtain a target product image, and convert the target product image into a grayscale image and a mask image.

[0070] In this embodiment, the target product image may be an image of the target product to be displayed; the target product image may be obtained by photographing the target product with an image acquisition device, or may be extracted from an original product display image corresponding to the target product. The original product display image corresponding to the target product refers to a low-quality product display image generated according to the prior art, for example, the original product display image is generated by directly inserting the product image into the background image.

[0071] After obtaining the target product image, the target product image is converted into a grayscale image and a mask image. Specifically, taking the target product image as a color image including three components of R, G, and B as an example, converting the target product image into a grayscale image means adjusting the values ​​of the three components of R, G, and B in the target product image to be the same. The conversion process may include: determining the mean or maximum value of the three components of R, G, and B in the target product image; adjusting the three components of R, G, and B in the target product image to the mean or the maximum value, and obtaining the grayscale image corresponding to the target product image. In addition, a convolutional neural network model for converting a color image into a grayscale image can be specially designed, and then the target product image can be converted into a grayscale image through the model.

[0072] Converting the target product image into a mask image includes: performing edge detection on the target product image through an edge detection algorithm (such as the Canny edge detection algorithm) to obtain an edge image; creating a full white image of the same size as the target product image as the initial mask image; sequentially traversing the pixels in the edge image, and each time a target pixel is traversed, determining whether the target pixel belongs to the edge according to the pixel value of the target pixel; if it does, setting the pixel corresponding to the target pixel in the initial mask image to black. Finally, a binary mask image is obtained.

[0073] S102: Acquire a target background image, and extract a low-frequency feature map representing light information from the target background image.

[0074] In the embodiment of the present invention, the target background image can be a background image newly produced according to the needs of product display, or it can be a background image extracted from the original product display image corresponding to the target product. The original product display image corresponding to the target product refers to a low-quality product display image generated according to the prior art, for example, the original product display image is generated by directly inserting the product image into the background image.

[0075] In the embodiment of the present invention, in order to ensure that the finally generated product display image can effectively attract consumers' attention and improve the purchase conversion rate, the target background image often contains rich and diverse information, including high-frequency details and low-frequency features (such as the overall light and shadow distribution, the approximate light and dark contrast, etc.). Therefore, when extracting the low-frequency feature map representing the light information from the target background image, the high-frequency details that are not important for the light information extraction can be weakened by gradually reducing the resolution, thereby extracting the low-frequency feature map representing the light information in the target background image.

[0076] In an optional implementation, extracting a low-frequency feature map representing light information from a target background image includes: extracting a low-frequency feature map representing light information from the target background image through a pre-trained light information extraction model; wherein the light information extraction model includes a plurality of light extractors connected in series; and the structure of any light extractor includes a first encoding layer, an intermediate layer, a first decoding layer, and a second decoding layer formed by a convolutional network. In a specific implementation, the target background image is gradually reduced in resolution through the plurality of light extractors of the light information extraction model to extract a low-frequency feature map representing light information.

[0077] In order to introduce in detail the process of extracting low-frequency feature maps through the light information extraction model, the light information extraction model includes 6 series-connected light extractors as an example. For any light extractor in the light information extraction model, the first encoding layer of the light extractor exemplarily uses 3 8*8 convolutional layers, the middle layer exemplarily uses 1 4*4 convolutional layer, the first decoding layer exemplarily uses 3 8*8 convolutional layers, and the second decoding layer exemplarily uses 3 16*16 convolutional layers. The specific structure of a single light extractor can be found in Figure 1b On this basis, the process of extracting low-frequency feature maps through a light information extraction model including 6 light extractors can be seen in Figure 1c . The target background image is exemplarily an image with a resolution of 1024*1024. The target background image is first input to the second decoding layer of the first light extractor, and then passes through the first decoding layer, the middle layer and the first encoding layer in sequence; and the output of the first encoding layer is used as the input of the second decoding layer of the next light extractor, and so on. After passing through 6 light extractors, a low-frequency feature map with a resolution of 16*16 can be obtained.

[0078] For further information, see Figure 1d, which shows the specific process of obtaining low-frequency feature maps through 6 light extractors. After the target background image with a resolution of 1024*1024 passes through the first light extractor, a feature map with a resolution of 512*512 is obtained; the feature map with a resolution of 512*512 is input to the second light extractor, and the second light extractor outputs a feature map with a resolution of 256*256; the feature map with a resolution of 256*256 is input to the third light extractor, and its output is a feature map with a resolution of 128*128; the feature map with a resolution of 128*128 is input to the fourth light extractor, and its output is a feature map with a resolution of 64*64; the feature map with a resolution of 64*64 is input to the fifth light extractor, and its output is a feature map with a resolution of 32*32; the feature map with a resolution of 32*32 is input to the sixth light extractor, and its output is a feature map with a resolution of 16*16; finally, the 16*16 feature map is used as the low-frequency feature map representing the light information output by the light information extraction model.

[0079] It is understandable that the above-mentioned method based on deep learning to extract the low-frequency feature map representing the light information in the target background image is only an optional implementation method, and other methods can also be selected to extract the low-frequency feature map. For example, through computer vision methods, the extraction process can be as follows: using feature extraction algorithms such as SIFT, SURF, and ORB, extract feature points and feature vectors in the target background image, and determine the changes in light in the image based on the distribution and intensity information of the feature points.

[0080] S103, generating a diffusion model through the image, and generating a product display image according to the grayscale image, the mask image, and the low-frequency feature map.

[0081] In an embodiment of the present invention, the image generation diffusion model is a generative model that can generate images with highly realistic details and rich semantic information based on input information. Exemplarily, the image generation diffusion model can be a DiffusionTransformer model. When generating a product display image, the grayscale image, the mask image, and the low-frequency feature map can be directly used as the input of the image generation diffusion model, and the product display image can be obtained according to the model output. It can be understood that the image generation diffusion model can determine the lighting relationship between the background image and the product image based on the input information, and then generate the final product display image with reference to this lighting relationship, thereby improving the quality of the generated product display image.

[0082] In some embodiments, the grayscale information can be first extracted from the grayscale image through an auxiliary image generation model; wherein the auxiliary image generation model can be a deep ControlNet model, and the extracted grayscale information can reflect the characteristics of the target product image, such as the light and dark distribution and contrast. The low-frequency feature map is processed through the cross-attention mechanism. It can be understood that the light information of the target background image may be more complicated, and not all low-frequency feature maps representing light information play an equally important role in generating an ideal new product display image. The cross-attention layer can filter the low-frequency feature map according to certain rules to identify key light information that has a greater impact on the final display effect, such as light features that can highlight the main body of the product and create a specific atmosphere. After filtering out the key light information, the cross-attention layer will further highlight these important light features, so that they can play a more dominant role in the subsequent injection of the image generation diffusion model and the final generation of the new image, and avoid being covered by other relatively minor light features or other types of features. On the basis of the above, the image generation diffusion model is used to generate a product display image according to the grayscale image, the mask image, and the low-frequency feature map, including: the grayscale information extracted from the grayscale image and the processed low-frequency feature map are used as image generation conditions, and input into the image generation diffusion model; the image generation diffusion model generates a product display image according to the image generation conditions and based on the input mask image.

[0083] In the embodiment of the present invention, the required product display image is generated by utilizing the lighting information of the background image and the product image, so that the fusion of the product image and the background image in the product display image is more natural, thereby significantly improving the quality of the product display image.

[0084] Embodiment 2

[0085] Figure 2 A flowchart of an image generation method is provided for an embodiment of the present invention. Figure 2 , the method comprises the following steps:

[0086] S201, obtaining an original product display image, and performing foreground separation processing on the original product display image through a semantic segmentation model to obtain an original foreground image and an original background image.

[0087] In this embodiment, the original product display image may be a low-quality image produced using existing technology. The semantic segmentation model is a model for classifying each pixel in the image according to its semantic category, and the SAM semantic segmentation model may be selected. The original product display image is input into the semantic segmentation model for foreground separation processing to obtain the original foreground image and the original background image.

[0088] It is understandable that since the quality of the original product display image is low, the background and foreground of the original product display image can be separated first, and then a high-quality product display image can be generated according to the image generation method proposed in this application, thereby optimizing the quality of the original product display image.

[0089] S202: Using the original foreground image as the target product image and the original background image as the target background image.

[0090] In this embodiment, the original foreground image is used as the target product image, and the original background image is used as the target background image, so as to generate a new product display image according to steps S203-S205.

[0091] S203: Obtain a target product image, and convert the target product image into a grayscale image and a mask image.

[0092] S204 , obtaining a target background image, and extracting a low-frequency feature map representing light information from the target background image.

[0093] In some embodiments, a low-frequency feature map representing light information can be extracted from a target background image through a pre-trained light information extraction model; wherein the light information extraction model includes a plurality of light extractors connected in series; the structure of any light extractor includes a first encoding layer, an intermediate layer, a first decoding layer, and a second decoding layer formed by a convolutional network.

[0094] S205, generating a diffusion model through the image, and generating a product display image according to the grayscale image, the mask image, and the low-frequency feature map.

[0095] In some embodiments, grayscale information is extracted from a grayscale image through an auxiliary image generation model; a low-frequency feature map is processed through a cross-attention mechanism; the grayscale information extracted from the grayscale image and the processed low-frequency feature map are used as image generation conditions and input into an image generation diffusion model; the image generation diffusion model generates a product display image according to the image generation conditions and based on the input mask image.

[0096] In the embodiment of the present application, the background image and the product image are first separated from the original product display image, and then the product display image is regenerated according to the light information in the background image, the grayscale information of the product image and the mask image corresponding to the product image. Since the newly generated product display image refers to the light information in the background image, the visual effect of the product display image can be effectively improved, thereby optimizing the quality of the original product display image.

[0097] Embodiment 3

[0098] Figure 3a The present invention provides a flowchart of an image generation method. Figure 3a , the method comprises the following steps:

[0099] S301: Construct training samples.

[0100] Each training sample includes a high-resolution sample image and a low-resolution sample image, and the image content of the high-resolution sample image and the low-resolution sample image is the same. For example, a training sample may include a low-resolution sample image with a resolution of 16*16 and a high-resolution sample image with a resolution of 1024*1024.

[0101] S302: For any training sample, add noise to a low-resolution sample image in the training sample to obtain a noise image.

[0102] In this embodiment, the obtained noise image is also an image with a resolution of 16*16.

[0103] S303: Input the noisy image into the light information extraction model for denoising and image generation processing to obtain a target image with the same resolution as the high-resolution sample image.

[0104] In the embodiment of the present invention, the light information extraction model exemplarily includes 6 light extractors, and the structure of each light extractor includes a first encoding layer, an intermediate layer, a first decoding layer, and a second decoding layer. On this basis, see Figure 3b , which shows the process of generating a target image from a noise image through a light information extraction model. Specifically, a 16*16 noise image is input to the first encoding layer of the first light extractor, and the second decoding layer of the first light extractor outputs a 32*32 image; the output of the first light extractor (32*32 image) is used as the input of the second light extractor; similarly, the output of the second light extractor (64*64 image) is used as the input of the third light extractor, and finally the output of the sixth light extractor (1024*1024 image) is used as the target image.

[0105] S304: Calculate the loss according to the high-resolution sample image and the target image, and adjust the parameters of the light information extraction model according to the loss calculation result.

[0106] In the embodiment of the present invention, based on the preset loss function, the loss between the high-resolution sample image and the target image is calculated, and the model parameters are adjusted according to the loss. In this way, multiple rounds of training are performed according to the above steps to obtain the final light information extraction model.

[0107] S305: Acquire a target product image, and convert the target product image into a grayscale image and a mask image.

[0108] S306 , obtaining a target background image, and extracting a low-frequency feature map representing light information from the target background image through a pre-trained light information extraction model.

[0109] S307, generating a diffusion model through the image, and generating a product display image according to the grayscale image, the mask image, and the low-frequency feature map.

[0110] In the embodiment of the present invention, by training the light information extraction model, a low-frequency feature map representing the light information can be accurately extracted from the target background image, providing a guarantee for the subsequent generation of high-quality product display images.

[0111] Embodiment 4

[0112] Figure 4 Schematic diagram of the structure of an image generation device provided by an embodiment of the present invention. This embodiment is applicable to the scene of generating high-quality product display images. Figure 4 As shown, the device comprises:

[0113] The first processing module 401 is used to obtain a target product image and convert the target product image into a grayscale image and a mask image;

[0114] The second processing module 402 is used to obtain a target background image and extract a low-frequency feature map representing light information from the target background image;

[0115] The image generation module 403 is used to generate a diffusion model through an image, and generate a product display image according to a grayscale image, a mask image, and a low-frequency feature map.

[0116] In some embodiments, the second processing module 402 is specifically configured to:

[0117] A pre-trained light information extraction model is used to extract a low-frequency feature map representing light information from the target background image.

[0118] Among them, the light information extraction model includes multiple light extractors connected in series; the structure of any light extractor includes a first encoding layer, an intermediate layer, a first decoding layer and a second decoding layer composed of a convolutional network.

[0119] In some embodiments, the apparatus further includes a model training module for training the light information extraction model, and the model training module is specifically used to:

[0120] Constructing training samples; wherein each training sample includes a high-resolution sample image and a low-resolution sample image, and the image content of the high-resolution sample image and the low-resolution sample image is the same;

[0121] For any training sample, noise is added to the low-resolution sample image in the training sample to obtain a noise image;

[0122] The noisy image is input into the light information extraction model for denoising and image generation processing to obtain a target image with the same resolution as the high-resolution sample image;

[0123] The loss is calculated based on the high-resolution sample image and the target image, and the parameters of the light information extraction model are adjusted according to the loss calculation results.

[0124] In some embodiments, the apparatus further comprises:

[0125] A grayscale extraction module is used to extract grayscale information from a grayscale image by using an auxiliary image generation model;

[0126] The feature processing module is used to process low-frequency feature maps through a cross-attention mechanism.

[0127] In some embodiments, the image generation module 403 is specifically used to:

[0128] The grayscale information extracted from the grayscale image and the processed low-frequency feature map are used as image generation conditions and input into the image generation diffusion model;

[0129] The image generation diffusion model generates a product display image based on the input mask image according to the image generation conditions.

[0130] In some embodiments, the apparatus further comprises:

[0131] A foreground separation module is used to obtain the original product display image and perform foreground separation processing on the original product display image through a semantic segmentation model to obtain an original foreground image and an original background image;

[0132] The determination module is used to use the original foreground image as the target product image and the original background image as the target background image.

[0133] The image generating device provided in the embodiment of the present invention can execute the image generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0134] Embodiment 5

[0135] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to be a controller device in a new energy vehicle. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0136] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0138] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as executing an image generation method.

[0139] In some embodiments, the image generation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image generation method in any other appropriate manner (e.g., by means of firmware).

[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The computer programs for implementing the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable image generating device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0142] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0145] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image generation method, characterized in that: include: Obtain a target product image, and convert the target product image into a grayscale image and a mask image; Acquire a target background image, and extract a low-frequency feature map representing light information from the target background image; A diffusion model is generated through an image, and a product display image is generated according to the grayscale image, the mask image, and the low-frequency feature map.

2. The method according to claim 1, characterized in that The step of extracting a low-frequency feature map representing light information from the target background image includes: Extracting a low-frequency feature map representing light information from the target background image by using a pre-trained light information extraction model; Wherein, the light information extraction model includes a plurality of light extractors connected in series; the structure of any of the light extractors includes a first encoding layer, an intermediate layer, a first decoding layer and a second decoding layer formed by a convolutional network.

3. The method according to claim 2, characterized in that The process of training the light information extraction model includes: Constructing training samples; wherein each training sample includes a high-resolution sample image and a low-resolution sample image, and the image contents of the high-resolution sample image and the low-resolution sample image are the same; For any training sample, adding noise to a low-resolution sample image in the training sample to obtain a noise image; Inputting the noise image into the light information extraction model for denoising and image generation processing to obtain a target image with the same resolution as the high-resolution sample image; The loss is calculated according to the high-resolution sample image and the target image, and the parameters of the light information extraction model are adjusted according to the loss calculation result.

4. The method according to claim 1, characterized in that The method further comprises: Extracting grayscale information from the grayscale image by using an auxiliary image generation model; The low-frequency feature maps are processed through a cross-attention mechanism.

5. The method according to claim 4, characterized in that The step of generating a diffusion model through an image and generating a product display image according to the grayscale image, the mask image, and the low-frequency feature map includes: Using the grayscale information extracted from the grayscale image and the processed low-frequency feature map as image generation conditions, and inputting them into the image generation diffusion model; The image generation diffusion model generates a product display image based on the input mask image according to the image generation condition.

6. The method according to claim 1, characterized in that The method further comprises: Obtain an original product display image, and perform foreground separation processing on the original product display image through a semantic segmentation model to obtain an original foreground image and an original background image; The original foreground image is used as the target product image, and the original background image is used as the target background image.

7. An image generating device, characterized in that: include: A first processing module is used to obtain a target product image and convert the target product image into a grayscale image and a mask image; A second processing module is used to obtain a target background image and extract a low-frequency feature map representing light information from the target background image; The image generation module is used to generate a diffusion model through an image, and generate a product display image according to the grayscale image, the mask image, and the low-frequency feature map.

8. The device according to claim 7, characterized in that The second processing module is specifically used for: Extracting a low-frequency feature map representing light information from the target background image by using a pre-trained light information extraction model; Wherein, the light information extraction model includes a plurality of light extractors connected in series; the structure of any of the light extractors includes a first encoding layer, an intermediate layer, a first decoding layer and a second decoding layer formed by a convolutional network.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 6 when executed.

Images