Method and apparatus for generating advertising image, and storage medium

Abstract

To provide a method and an apparatus for generating an advertising image, and a storage medium.SOLUTION: A method includes the steps of: generating a text control parameter for an AI image generative model on the basis of text descriptions of an advertising image of a product; generating an image space control parameter for the AI image generative model on the basis of a product image having a foreground representing an image block of the product and a background showing a single color; generating a noise image for the AI image generative model on the basis of the product image; generating an initial product image, using the AI image generative model, on the basis of the text control parameter, the image space control parameter, and the noise image; generating a restored product image by replacing a corresponding region in the initial product image with the image block; and generating an advertising image of the product by executing post-processing that merges the foreground image and the background image on the restored product image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to image processing, and specifically, to a method for generating an advertisement image executed by a computer, an apparatus for generating an advertisement image, and a computer-readable non-transitory storage medium storing a program.

Background Art

[0002] With the development of computer science and artificial intelligence, it has become increasingly common and effective to realize information processing by using a computer to execute an artificial intelligence (AI) model based on a neural network. Image generation is one of the important application fields of the artificial intelligence model.

[0003] A product advertisement including a product image shows the features and effects of the target product, helps in promoting the product, improves the recognition of the product, and further improves the sales and profits of the product. In conventional advertisement production, it was necessary to manually create a product image showing the features of the product as an advertisement image for the product. A large amount of labor and time were required in the production of conventional advertisement images. Even when creating an advertisement image based on a basic template, a large amount of labor and time were still required.

[0004] Currently, there is a method for generating an image based on a generative AI model. Such an AI model can generate creative images but cannot restore the details of a product image. Also, in the case of particularly rare products, existing AI models usually cannot generate an advertisement image that matches the product (product image block) within the product image of the product.

[0005] It is desirable to generate a high-quality advertisement image of a product.

Summary of the Invention

Problems to be Solved by the Invention

[0006] The following provides a brief overview of the present disclosure to facilitate a basic understanding of its aspects. Note that this brief overview is not an exhaustive overview of the present disclosure, nor is it intended to specifically identify key points or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Instead, it aims to simply explain concepts in a simple form as a prelude to the more detailed description that follows.

[0007] The inventors of the present invention have studied image generation models and image processing technologies and propose an improved method for generating advertising images that expects advantages in efficiency and image quality.

Means for Solving the Problems

[0008] In one aspect of the present disclosure, there is provided a method for generating an advertising image, which is executed by a computer, and includes: generating text control parameters for an AI image generation model based on a text description of an advertising image of a product; generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color; generating a noise image for the AI image generation model based on the product image; using the AI image generation model to generate an initial product image based on the text control parameters, the image space control parameters, and the noise image; generating a restored product image by replacing a corresponding area in the initial product image with the image block; and generating the advertising image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image.

[0009] In one aspect of the present disclosure, there is provided an apparatus for generating an advertising image, which includes: a memory storing instructions; and at least one processor configured to execute the instructions to implement the above method for generating an advertising image.

[0010] In another aspect of the present disclosure, there is provided a computer-readable non-transitory storage medium storing a program, which, when executed by a computer, causes the computer to perform the steps of: generating text control parameters for an AI image generation model based on a text description of an advertisement image of a product; generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color; generating a noise image for the AI image generation model based on the product image; generating an initial product image using the AI image generation model based on the text control parameters, the image space control parameters, and the noise image; generating a restored product image by replacing a corresponding region in the initial product image with the image block; and generating the advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image.

[0011] The advantageous effects of the method, apparatus, and storage medium of the present disclosure include at least one of the following advantageous effects: improving the creation efficiency of advertisement images, improving the quality of advertisement images, fully expressing the details of products, and improving the consistency between the products in the advertisement images and the actual products.

Brief Description of the Drawings

[0012] In order to more easily understand the above and other objects, features, and advantages of the present disclosure, the embodiments of the present disclosure will be described below with reference to the drawings. It should be noted that the drawings are only for explaining the principle of the present disclosure. In the drawings, it is not necessary to draw the sizes and relative positions of each part according to the scale. The same reference numerals may represent the same features.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0013] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. For convenience of explanation, not all features of the actual embodiments are shown in the specification. It should be noted that when those skilled in the art implement the embodiments, they may make specific decisions to implement the embodiments, and these decisions may be changed according to the embodiments.

[0014] Note that, for clarity of the present disclosure, only the components of the device and / or the processing steps closely related to the present disclosure are shown in the drawings, and details not related to the present disclosure are omitted.

[0015] Note that this disclosure is not limited to the described embodiments and will be described below with reference to the accompanying drawings. In this specification, when feasible, examples may be combined with each other, features of different examples may be replaced or utilized, or one or more features may be omitted in one example.

[0016] The code of the computer program for performing the operations of each aspect of the exemplary embodiments disclosed herein may be described in any combination of one or more programming languages, which include object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as the "C" programming language or similar programming languages.

[0017] The method of this disclosure may also be implemented by a circuit having a corresponding functional configuration. The circuit includes a circuit for a processor.

[0018] One aspect of this disclosure relates to a method for generating an advertisement image. The method may be implemented by a computer. The following will exemplarily describe the method with reference to FIG. 1.

[0019] FIG. 1 is a flowchart showing an example of an advertisement image generation method 100 according to one embodiment of this disclosure.

[0020] In step Op101, based on the text description txt of the advertisement image of the product prdt, a text control parameter c_txt for the AI image generation model imM is generated. For example, the text control parameter c_txt may be a vector having a predetermined number of elements.

[0021] For example, the product prdt may be a bottled beverage having specified specifications and a specified package. The product has various features regarding details, such as the color, material, shape, height, transparency, pattern, texture, label, pattern, etc. of the container (bottle).

[0022] The text description txt may include a description of the desired background and foreground in the generated image, where the description of the foreground is related to the product prdt. For example, if the product prdt is a bottled beverage, the text description txt may include the noun "bottle".

[0023] An exemplary text description txt for a bottled beverage may be "A close-up of the crystal-clear bottle of water floating in mid-air against a backdrop of a tranquil waterfall".

[0024] As an example, the text description txt may be characters manually input by the user. As an example, the text description txt may be generated by voice recognition software recognizing what is orally input by the user. As another example, the text description txt may be generated using a language model. For example, the language model may be ChatGPT (Chat Generative Pre-trained Transformer, i.e., a natural language processing tool based on artificial intelligence technology developed by OpenAI). ChatGPT is a prior art and its description is omitted here.

[0025] Various text encoders may be used to convert the text description txt into the text control parameter c_txt. As an example, a CLIP text encoder may be used to generate the text control parameter c_txt based on the text description txt. For the CLIP text encoder, reference may be made to the paper "Learning Transferable Visual Models From Natural Language Supervision" by Alec Radford et al. shown in the following link.

[0026] https: / / arxiv.org / pdf / 2103.00020.pdf。

[0027] The CLIP text encoder is a conventional technique, and its description is omitted here.

[0028] The AI image generation model imM can generate images based on text control parameters, image space control parameters, and noise images. For example, the AI image generation model imM may be ControlNet. For ControlNet, for example, refer to the paper "Adding Conditional Control to Text-to-Image Diffusion Models" by Lvmin Zhang et al. The link to the paper is as follows.

[0029] https: / / arxiv.org / pdf / 2302.05543.pdf。

[0030] ControlNet is a conventional technique, and its description is omitted here. Note that the AI image generation model imM may be an image generation model customized by the user with reference to ControlNet. The customized model may be, for example, a model trained using samples desired by the user.

[0031] In step Op103, based on the product image F where the foreground is the image block Bp showing the product prdt and the background is a single color Cbg, an image space control parameter c_prdt for the AI image generation model imM is generated. The image block Bp may be obtained from the original product image Imo provided by the user. The image block Bp contains information about the details of the actual product. The image block Bp may be obtained by the user dividing the image where the product is to be displayed in the advertisement. The original image block Imo preferably has a resolution of a predetermined resolution or higher. The original image block Imo may be, for example, an image obtained by photographing the actual product at a predetermined angle, under a predetermined illumination, and against a predetermined background. The original image block Imo may be, for example, a manually drawn image or an image drawn manually using drawing software. The shape of the image block Bp is determined by the contour line of the product prdt in the original image block Imo.

[0032] In step Op105, based on the product image F, a noise image z p,r for the AI image generation model imM is generated.

[0033] In step Op107, using the AI image generation model imM, based on the text control parameter c_txt, the image space control parameter c_prdt, and the noise image z p,r , an initial product image gen ori is generated. When generating the initial product image gen ori using ControlNet, the generation operation may be expressed as in Equation (1).

[0034]

Equation

[0035] In step Op109, by replacing the corresponding area in the initial product image gen ori with the image block Bp, a restored product image gen prdt is generated. That is, to replace or substantially replace the product image block in the initial product image, the image block Bp is pasted onto the corresponding area in the image generated by the model.

[0036] In step Op111, an advertisement image I_final of the product prdt is generated by performing post - processing post_Pr of fusing the foreground image and the background image on the restored product image gen prdt The advertisement image generation method 100 may be configured to automatically output an advertisement image of a product based on the input text description txt and the original product image Imo. Therefore, the efficiency of creating the advertisement image can be improved, the innovativeness of the advertisement image can be improved, and the labor cost and time cost can be reduced. In the advertisement image generation method 100, the product image block in the final advertisement image is not completely generated by the image model, and basically can be regarded as the original product image from the product desired by the user. Therefore, it can be ensured that the product in the advertisement image matches the actual product and fully expresses the details and features of the actual (true) product. According to the post - processing, the synthesized final advertisement image is located near the contour line of the product, and the transition between the foreground and the background is more natural and looks more real. That is, the advertisement image generation method 100 can improve the quality of the advertisement image in terms of consistency, details, and vividness, and improve the effect of the advertisement. Overall, the advertisement image generation method 100 can efficiently generate the advertisement image and ensure the high quality of the advertisement image.

[0037]

[0038] The following further explains the details of each operation in the advertisement image generation method 100.

[0039] In one embodiment, the step of generating the text control parameter c_txt for the AI image generation model includes the step of generating the text description txt using a language model, and the step of generating the feature vector of the text description as the text control parameter c_txt using a text encoder. The language model includes, but is not limited to, ChatGPT. The text encoder includes, but is not limited to, the CLIP text encoder.

[0040] Figure 2 is a flowchart showing an example of the product image F generation method 200 according to one embodiment of the present disclosure. The advertisement image generation method 200 includes steps Op201, Op203, and Op205.

[0041] In step Op201, by dividing the original image Imo of the product prdt provided by the user, the foreground image Bp' showing the product prdt in the original image Imo is extracted. As an example, a segmentation model such as the model SAM (Segment Anything Model) is used to extract the foreground image Bp' of the product prdt from the original image Imo. The segmentation model SAM is a prior art. For example, refer to the paper "Segment Anything" by Alexander Kirillov et al. provided by the following link.

[0042] https: / / arxiv.org / pdf / 2304.02643.pdf.

[0043] Figure 3 is a diagram showing an example of the original image Imo of a product according to one embodiment of the present disclosure. Here, for simplicity, features including patterns, labels, colors, etc. are omitted, and the original image of the product in a bottle is shown in a grayscale diagram. Note that the original image of the product actually used is usually a color image and includes details such as labels, trademarks, patterns, etc.

[0044] In step Op203, a product image block Bp is generated by scaling (enlarging / reducing) the foreground image Bp' based on a predetermined image size S. The predetermined image size S is, for example, the size of the advertisement image Im_final desired by the user of the product prdt. Usually, in the advertisement image, in order to display the overall image of the product in the original product image, the maximum width w of the image block Bp is scaled so as to be less than or equal to the width W of the advertisement image Im_final (e.g., w < W, w ≤ 0.80W, w ≤ 0.62W, w ≤ 0.50W, or w ≤ 0.40W), and the maximum height h of the image block Bp is scaled so as to be less than or equal to the height H of the advertisement image Im_final (h < H, h ≤ 0.80H, h ≤ 0.62H, h ≤ 0.50H, or h ≤ 0.40H). Step Op203 may be omitted. For example, if the size of the foreground image Bp' is suitable for pasting on the advertisement image Im_final of the predetermined size of the product prdt being displayed, the foreground image Bp' may be used as the product image block Bp as it is.

[0045] In step Op205, a product image F is generated based on the image block Bp, a single color Cbg, and a predetermined position p(x, y) of the image block Bp. The predetermined position p(x, y) may be the coordinates of the upper left corner of the circumscribed frame of the image block Bp in the product image F.

[0046] FIG. 4 is a diagram showing an example of a product image F according to one embodiment of the present disclosure. The product image F includes an image block Bp as a foreground and a white background (i.e., the single color Cbg is "white"). The image block Bp is an image block combining the bottle body and the bottle cap, and may be obtained by dividing the original image Imo. Note that the single color Cbg includes, but is not limited to, "white". Here, for simplicity, the product image F is shown as a grayscale image. In an actual application scenario, the product image F is usually a color image.

[0047] In one embodiment, in the advertising image generation method 100, the step of generating the image space control parameter c_prdt for the AI image generation model imM includes the step of generating an edge image Fe by performing edge detection on the product image F, and the step of generating the features of the product image F as the image space control parameter c_prdt by encoding the edge image Fe with the neural network f(pi,φ). pi is the input of the neural network f (which may be Fe here), and φ is the configuration parameter of the neural network f. As an example, edge detection is performed using the Canny edge detection algorithm. The Canny edge detection algorithm is a prior art, for example, the Canny detection function included in the cross-platform computer vision and machine learning software library OpenCV, and for example, reference may be made to the description described in the following paper.

[0048] A Computational Approach To Edge Detection; JOHN CANNY, IEEE Transactions on Pattern Analysis and Machine Intelligence; Volume: PAMI-8, Issue: 6, November 1986; page(s)679-698。

[0049] FIG. 5 is a diagram showing an example of an edge image Fe according to one embodiment of the present disclosure. As can be seen from FIG. 5, the edge image provides a lot of edge information including the contour line of the product. This is useful when generating a product image using an image generation model.

[0050] In one embodiment, in the advertising image generation method 100, the step of generating the noise image z p,r includes generating a product noise image z p by repeatedly and gradually adding noise to the product image F, and a random noise image z rgenerating and, based on a mask msk indicating a coverage area of an image block Bp in a product image F, a product noise image z p and a random noise image z r and combining them to obtain a noise image z p,r The mask msk is, for example, a two-dimensional matrix corresponding to the product image F, and an element m i,j of the matrix is determined according to Equation (2).

[0051]

Equation

[0052] The noise image z p,r may be obtained by combining the product noise image z p and the random noise image z r according to Equation (3).

[0053]

Equation

[0054] FIG. 6 is a diagram showing an example of an image corresponding to the mask msk. Here, the mask is shown as a black-and-white image. In FIG. 6, the contour line (the position of the transition between white and black) of the image block (Bp) composed of the combination of the bottle body and the bottle cap can be recognized. That is, the mask msk can indicate the positions of the pixel points on the contour line of the image block Bp in the product image F. By replacing the corresponding region in the initial product image with the image block Bp based on the mask, a restored product image may be generated.

[0055] In one embodiment, in the advertisement image generation method 100, the step of generating the restored product image gen prdt includes the step of combining the product image F and the initial product image gen ori based on a mask indicating the coverage area of the image block Bp in the product image F to obtain a restored product image. For example, the restored product image gen prdt is synthesized according to Equation (4).

[0056]

Number

[0057] FIG. 7 is a diagram showing an example of an initial product image according to an embodiment of the present disclosure. As can be seen by comparing FIG. 4 and FIG. 7, the product in the initial product image generated by the model does not match the product in the original product image, and there are differences in details.

[0058] In one embodiment, in the advertisement image generation method 100, the AI image generation model imM is a noise image zp,r The initial product image gen is configured to be generated by gradually removing noise from ori it.

[0059] In one embodiment, in the method 100 for generating an advertisement image, the post-processing may include post-processing of Poisson synthesis, post-processing of boundary transition smoothing, and post-processing of Gaussian filtering smoothing.

[0060] The post-processing of Poisson synthesis includes performing Poisson image synthesis on the restored product image and the image block to obtain a Poisson synthesis image Img possion . Poisson synthesis is a conventional image processing technique. For example, reference may be made to the paper "Poisson Image Editing" by Patrick Perez et al. provided at the following link.

[0061] https: / / www.cs.jhu.edu / ~misha / Fall07 / Papers / Perez03.pdf.

[0062] FIG. 8 is a diagram showing an example of a Poisson synthesis image according to one embodiment of the present disclosure. Compared with the restored product image, the Poisson synthesis image looks more natural and realistic in the vicinity of the contour line of the product and has fewer traces of synthesis. Here, the Poisson synthesis image is shown as, for example, a grayscale image. In actual applications, the Poisson synthesis image is usually a color image.

[0063] The post-processing of boundary transition smoothing includes performing boundary transition smoothing on the inner region inside the contour line of the product in the Poisson synthesis image Img possion based on the boundary of the image block Bp in the product image F to obtain a first smoothed image Img1. Here, the first smoothed image Img1 may be determined according to Equation (5).

[0064]

Equation

[0065] The post - processing of Gaussian filtering smoothing includes performing Gaussian filtering smoothing on the contour region near the contour line of the product in the first smoothed image Img1 to obtain the second smoothed image Img2 as the product advertisement image I_final. The second smoothed image Img2 may be determined according to Equation (6).

[0066]

Equation

[0067] FIG. 9 is a diagram showing an example of a second smoothed image according to an embodiment of the present disclosure. Compared with the restored product image, the second smoothed image has a smoother and more natural transition near the contour line of the product, and the quality of the synthetic image is improved. Here, the second smoothed image is shown as a grayscale image, for example. In actual applications, the second smoothed image is usually a color image.

[0068] Note that in the above embodiments, it is shown that three post-processings are sequentially executed. However, when the requirement for image quality is not so high, only one or two of them may be executed. Also, in the present disclosure, the post-processing may include other edge smoothing processes for the image region.

[0069] In one embodiment of the present disclosure, a generating device for an advertisement image is provided. The following is an explanation of an example of the device with reference to FIG. 10.

[0070] FIG. 10 is a block diagram showing an example of a generating device 1000 for an advertisement image according to an embodiment of the present disclosure.

[0071] The advertisement image generation device 1000 includes a memory 1001 in which instructions Inst are stored, and at least one processor 1003 connected to the memory 1001 and configured to execute instructions so as to implement the advertisement image generation method 100. For example, the at least one processor 1003 is configured to execute instructions to implement steps including: generating text control parameters for an AI image generation model based on a text description of an advertisement image of a product; generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color; generating a noise image for the AI image generation model based on the product image; generating an initial product image using the AI image generation model based on the text control parameters, the image space control parameters, and the noise image; generating a restored product image by replacing corresponding regions in the initial product image with image blocks; and generating an advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image.

[0072] In another aspect of the present disclosure, there is provided a computer-readable non-transitory storage medium storing a program. When the program is executed by a computer, the computer is caused to perform steps of generating text control parameters for an AI image generation model based on a text description of an advertisement image of a product, generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color, generating a noise image for the AI image generation model based on the product image, generating an initial product image using the AI image generation model based on the text control parameters, the image space control parameters, and the noise image, generating a restored product image by replacing corresponding regions in the initial product image with image blocks, and generating an advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image. Further details of the program may refer to the description of the advertisement image generation method 100.

[0073] In one aspect of the present disclosure, there is further provided an information processing apparatus.

[0074] FIG. 11 is a block diagram showing an example of an information processing apparatus 1100 according to an embodiment of the present invention. In FIG. 11, a central processing unit (CPU) 1101 executes various processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage unit 1108 into a random access memory (RAM) 1103. The RAM 1103 stores data necessary for the CPU 1101 to execute various processes as needed.

[0075] The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output interface 1105 is also connected to the bus 1104.

[0076] The input unit 1106 (including a keyboard, a mouse, etc.), the output unit 1107 (including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), the storage unit 1108 (including, for example, a hard disk, etc.), and the communication unit 1109 (including a network interface card, such as a LAN card, a modem, etc.) are connected to the input / output interface 1105. The communication unit 1109 executes communication processing via a network, such as the Internet, a local area network, a mobile network, or a combination thereof.

[0077] Optionally, the driver 1110 may be connected to the input / output interface 1105. The removable medium 1111 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., and is set up in the driver 1110 as needed, and the computer program read from it is installed in the storage unit 1108 as needed.

[0078] The CPU 1101 may execute a program corresponding to the method for generating an advertisement image.

[0079] The advantageous effects of the method, apparatus, and storage medium according to the present disclosure include at least one of the following advantageous effects. The generation efficiency of the advertisement image can be improved. The quality of the advertisement image can be improved. The details of the product can be fully represented. The consistency between the product in the advertisement image and the actual product can be improved. In particular, even if the product in the original product image is rare, an advertisement image more consistent with the product in the original product image can be generated.

[0080] As described above, in the present disclosure, the principle of improving the generation process of the advertisement image is explained. Note that the effects of the present disclosure are not necessarily limited to the above effects, and in addition to or instead of the effects described in the above paragraphs, any of the effects shown in this specification can be obtained, or other effects can be understood from this specification.

[0081] The above has described specific embodiments of the present disclosure. However, those skilled in the art can make various changes (in the case of rows, combining or replacing the features of each embodiment), improvements or equivalents to the present disclosure within the gist and scope of the appended claims. These changes, improvements or equivalents belong to the protection scope of the present disclosure. For example, the shelves in the space may be replaced with other obstacles that block the target passage.

[0082] In addition, the terms "comprising" and "having" mean the presence of the features, elements, steps or members described in this specification, but do not exclude the presence or addition of one or more other features, elements, steps or members.

[0083] Furthermore, the methods of each embodiment of the present invention are not limited to being executed in the chronological order described in the specification or shown in the drawings. They may be executed in other chronological orders, or may be executed in parallel or independently. Therefore, the execution order of the methods described in this specification does not limit the technical scope of the present invention.

[0084] Also, regarding the embodiments including the above-described each embodiment, the following additional notes are disclosed, but are not limited to these additional notes. (Additional Note 1) A method for generating an advertising image executed by a computer, comprising: generating text control parameters for an AI image generation model based on a text description of an advertising image of a product; generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color; generating a noise image for the AI image generation model based on the product image; using the AI image generation model to generate an initial product image based on the text control parameters, the image space control parameters and the noise image; Generating a restored product image by replacing a corresponding area in the initial product image with the image block; Generating the advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image, a method comprising the steps. (Appendix 2) The step of generating text control parameters for the AI image generation model comprises: Generating the text description using a language model; Generating a feature vector of the text description as the text control parameter using a text encoder, the method according to Appendix 1. (Appendix 3) When generating the product image: Extracting a foreground image showing the product in the original image by dividing the original image of the product provided by a user; Generating an image block of the product by scaling the foreground image based on a predetermined image size; Generating the product image based on the image block, the single color, and a predetermined position of the image block; The advertisement image has the predetermined image size, the method according to Appendix 1. (Appendix 4) The step of generating image space control parameters for the AI image generation model comprises: Generating an edge image by performing edge detection on the product image; Generating features of the product image as the image space control parameters by encoding the edge image with a neural network, the method according to Appendix 1. (Appendix 5) The step of generating the noise image comprises: Generating a product noise image by repeatedly and gradually adding noise to the product image; Generating a random noise image; A method according to Appendix 1, comprising: combining the product noise image and the random noise image based on a mask indicating a coverage area of the image block in the product image to obtain the noise image. (Appendix 6) The step of generating the restored product image includes A method according to Appendix 1, comprising: combining the product image and the initial product image based on a mask indicating a coverage area of the image block in the product image to obtain a restored product image. (Appendix 7) A method according to Appendix 1, wherein the AI image generation model is configured to generate the initial product image by gradually removing noise from the noise image. (Appendix 8) The post-processing includes obtaining a Poisson composite image by performing Poisson image synthesis on the restored product image and the image block; obtaining a first smoothed image by performing boundary transition smoothing on an inner area inside a contour line of the product in the Poisson composite image based on a boundary of the image block in the product image; A method according to Appendix 1, comprising: obtaining a second smoothed image as an advertisement image of the product by performing Gaussian filtering smoothing on a contour area near the contour line of the product in the first smoothed image. (Appendix 9) The inner area has a width indicated by a first smoothing threshold, according to the method of Appendix 8. (Appendix 10) The contour area has a width indicated by twice a second smoothing threshold, according to the method of Appendix 8. (Appendix 11) An apparatus for generating an advertisement image, comprising a memory storing instructions; at least one processor, wherein the at least one processor Generating text control parameters for an AI image generation model based on the text description of the advertisement image of the product; Generating image space control parameters for the AI image generation model based on a product image in which the foreground shows the product and the background is a single color; Generating a noise image for the AI image generation model based on the product image; Generating an initial product image using the AI image generation model based on the text control parameters, the image space control parameters, and the noise image; Generating a restored product image by replacing a corresponding area in the initial product image with the image block; Generating the advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image, an apparatus configured to execute the instructions so as to implement the above steps. (Appendix 12) The step of generating text control parameters for the AI image generation model includes: Generating the text description using a language model; Generating a feature vector of the text description as the text control parameters using a text encoder, the apparatus according to Appendix 11. (Appendix 13) When generating the product image, Extracting a foreground image showing the product in the original image by dividing the original image of the product provided by the user; Generating an image block of the product by scaling the foreground image based on a predetermined image size; Generating the product image based on the image block, the single color, and a predetermined position of the image block; The advertisement image has the predetermined image size, the apparatus according to Appendix 11. (Appendix 14) The step of generating the image space control parameter for the AI image generation model includes: generating an edge image by performing edge detection on the product image; generating the features of the product image as the image space control parameter by encoding the edge image with a neural network. The device according to Supplementary Note 11 (Supplementary Note 15) The step of generating the noise image includes: generating a product noise image by repeatedly and gradually adding noise to the product image; generating a random noise image; combining the product noise image and the random noise image based on a mask indicating the coverage area of the image block in the product image to obtain the noise image. The device according to Supplementary Note 11 (Supplementary Note 16) The step of generating the restored product image includes: combining the product image and the initial product image based on a mask indicating the coverage area of the image block in the product image to obtain the restored product image. The device according to Supplementary Note 11 (Supplementary Note 17) The AI image generation model is configured to generate the initial product image by gradually removing noise from the noise image. The device according to Supplementary Note 11 (Supplementary Note 18) The post-processing includes: obtaining a Poisson composite image by performing Poisson image synthesis on the restored product image and the image block; obtaining a first smoothed image by performing boundary transition smoothing on the inner region inside the contour line of the product in the Poisson composite image based on the boundary of the image block in the product image; By performing Gaussian filtering smoothing on the contour area near the contour line of the product in the first smoothed image, obtaining a second smoothed image as the advertisement image of the product, the apparatus according to Supplementary Note 11. (Supplementary Note 19) The inner region has a width indicated by a first smoothing threshold, The contour region has a width twice that indicated by a second smoothing threshold, the apparatus according to Supplementary Note 18. (Supplementary Note 20) A computer-readable non-transitory storage medium storing a program, when the program is executed by a computer, causing the computer to, Generating text control parameters for an AI image generation model based on a text description of an advertisement image of a product; Generating image space control parameters for the AI image generation model based on an image block in which the foreground shows the product and the background is a single color; Generating a noise image for the AI image generation model based on the product image; Generating an initial product image using the AI image generation model based on the text control parameters, the image space control parameters, and the noise image; Generating a restored product image by replacing a corresponding region in the initial product image with the image block; Generating the advertisement image of the product by performing post-processing of fusing a foreground image and a background image on the restored product image. A storage medium that causes the above steps to be executed.

Claims

1. 1. A computer implemented method for generating advertising images, comprising: generating text control parameters for an AI image generation model based on a text description of an advertising image of the product; generating image space control parameters for the AI ​​image generation model based on a product image in which the foreground is an image block showing the product and the background is a single color; generating a noise image for the AI ​​image generation model based on the product image; generating an initial product image based on the text control parameters, the image spatial control parameters and the noise image using the AI ​​image generation model; generating a reconstructed product image by replacing corresponding regions in the initial product image with the image blocks; and performing post-processing on the reconstructed product image to fuse a foreground image with a background image to generate an advertising image for the product.

2. The step of generating text control parameters for an AI image generation model includes: generating said text description using a language model; and generating a feature vector of the text description as the text control parameter using a text encoder.

3. When generating the product image, Extracting a foreground image showing the product in the original image provided by a user by segmenting the original image of the product; generating an image block of the product by scaling the foreground image based on a predetermined image size; generating the product image based on the image blocks, the single color, and the predetermined positions of the image blocks; The method of claim 1 , wherein the advertising image has the predetermined image size.

4. The step of generating image space control parameters for the AI ​​image generation model includes: generating an edge image by performing edge detection on the product image; and generating features of the product image as the image space control parameters by encoding the edge image with a neural network.

5. The step of generating a noise image includes: generating a product noise image by iteratively and incrementally adding noise to the product image; generating a random noise image; and combining the product noise image and the random noise image into the noise image based on a mask indicating coverage areas of the image blocks in the product image.

6. The step of generating a restored product image includes: The method of claim 1 , further comprising combining the product image and the initial product image into a reconstructed product image based on a mask indicating coverage areas of the image blocks in the product image.

7. The method of claim 1 , wherein the AI ​​image generation model is configured to generate the initial product image by progressively removing noise from the noisy image.

8. The post-treatment is performing Poisson image blending on the reconstructed product image and the image blocks to obtain a Poisson blended image; performing boundary transition smoothing on an inner region inside the contour of the product in the Poisson synthesis image based on boundaries of the image blocks in the product image to obtain a first smoothed image; and performing Gaussian filtering smoothing on a contour region in the first smoothed image adjacent to a contour line of the product to obtain a second smoothed image as an advertising image of the product.

9. An apparatus for generating an advertising image, comprising: A memory having instructions stored therein; and at least one processor configured to execute said instructions to implement a method according to any preceding claim.

10. A computer-readable non-transitory storage medium storing a program, the program being executed by a computer to cause the computer to: generating text control parameters for an AI image generation model based on a text description of an advertising image of the product; generating image space control parameters for the AI ​​image generation model based on a product image in which the foreground is an image block showing the product and the background is a single color; generating a noise image for the AI ​​image generation model based on the product image; generating an initial product image based on the text control parameters, the image spatial control parameters and the noise image using the AI ​​image generation model; generating a reconstructed product image by replacing corresponding regions in the initial product image with the image blocks; and performing post-processing on the reconstructed product image to fuse a foreground image with a background image to generate an advertising image for the product.

Images