Image processing method and system

Abstract

The present invention provides an image processing method and system, which obtains prompt words and advertisement text to be processed related to a target advertisement category; inputs the prompt words to be processed into an image generation model to obtain a first advertisement material image related to the target advertisement category; and calls a transformer and a diffusion model to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image. In this solution, prompt words and advertisement text to be processed related to the target advertisement category are obtained. The image generation model is used to generate a first advertisement material image related to the target advertisement category based on the prompt words to be processed, and then the transformer and the diffusion model are used to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image. The advertisement material image can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by designers, thereby improving the production efficiency of advertisement material images.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a picture processing method and system. Background Art

[0002] When making advertising material images, it is necessary to shoot the advertising base map according to actual needs, and then the designer will edit the advertising base map to obtain the final advertising material image.

[0003] However, on the one hand, it takes a lot of manpower and material resources to arrange the scene and shoot when shooting the advertising background map. On the other hand, designers also need to spend time to make secondary edits to the advertising background map. The above two aspects will lead to low production efficiency of advertising material images. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and system for processing an image to solve the problem of low production efficiency in the current method of producing advertising material images.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of an embodiment of the present invention discloses a method for processing an image, the method comprising:

[0007] Obtain pending prompt words and advertisement texts related to the target advertisement category;

[0008] Inputting the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category, wherein the image generation model is trained based on sample data corresponding to multiple advertising categories and a fine-tuning model;

[0009] The transformer and the diffusion model are called to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image.

[0010] Preferably, calling a transformer and a diffusion model to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image includes:

[0011] Using the first advertising material image, the position information variable and the size information variable, a first sequence is formed;

[0012] Performing word segmentation processing on the advertisement text to obtain a second sequence;

[0013] Inputting the first sequence and the second sequence into a transformer for encoding and decoding to obtain a text position and a text size related to the advertisement text;

[0014] Adding the advertisement text having a size equal to the text size to the text position of the first advertisement material picture to obtain a first intermediate picture;

[0015] Calling the diffusion model and Controlnet to extract edge information of the target object and the advertisement text in the first intermediate image to obtain a second intermediate image;

[0016] Creating a mask based on the text position;

[0017] The mask and the second intermediate image are input into unet to obtain a second advertising material image with the style of the advertising text adjusted.

[0018] Preferably, the first advertisement material image, the position information variable and the size information variable are used to form a first sequence, including:

[0019] Adjusting the size of the first advertising creative image;

[0020] Splitting the resized first advertising creative image into a plurality of image blocks to form corresponding image columns;

[0021] A position information variable and a size information variable are added to the picture sequence to obtain a first sequence.

[0022] Preferably, the prompt word to be processed is input into a preset picture generation model to obtain a first advertising material picture related to the target advertising category, including:

[0023] When the target advertisement category is a first advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement base image as a first advertisement material image related to the first advertisement category;

[0024] When the target advertisement category is a second advertisement category, inputting the prompt word to be processed into a preset image generation model to obtain an advertisement background image related to the second advertisement category;

[0025] The advertisement background image and the product image submitted by the user are merged and the image is illuminated to obtain a first advertisement material image.

[0026] Preferably, obtaining prompt words and advertisement texts to be processed related to the target advertisement category includes:

[0027] Acquire the advertisement description content and advertisement text related to the target advertisement category input by the user, wherein the advertisement text at least includes the advertisement title and / or advertisement selling point information;

[0028] The advertisement description content is optimized using a prompt word optimization algorithm to obtain prompt words to be processed.

[0029] Preferably, after obtaining the second advertising material image, the method further includes:

[0030] Adding a specified icon and a specified watermark to the second advertising creative image;

[0031] The second advertising creative image with the specified icon and the specified watermark added is delivered.

[0032] A second aspect of an embodiment of the present invention discloses an image processing system, the system comprising:

[0033] An acquisition unit, used for acquiring prompt words and advertisement texts to be processed that are related to the target advertisement category;

[0034] A processing unit, configured to input the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category, wherein the image generation model is trained based on sample data corresponding to a plurality of advertising categories and a fine-tuning model;

[0035] The editing unit is used to call the transformer and the diffusion model to edit the first advertising material picture based on the advertisement text to obtain the second advertising material picture.

[0036] Preferably, the editing unit includes:

[0037] A generating module, configured to form a first sequence using the first advertising material image, the position information variable and the size information variable;

[0038] A word segmentation module, used for performing word segmentation processing on the advertisement text to obtain a second sequence;

[0039] A processing module, configured to input the first sequence and the second sequence into a transformer for encoding and decoding, and obtain a text position and a text size related to the advertisement text;

[0040] An adding module, used for adding the advertisement text having a size equal to the text size to the text position of the first advertisement material picture to obtain a first intermediate picture;

[0041] An extraction module, used for calling a diffusion model and Controlnet to extract edge information of a target object and the advertisement text in the first intermediate image to obtain a second intermediate image;

[0042] A making module, used for making a mask according to the position of the text;

[0043] The adjustment module is used to input the mask and the second intermediate image into unet to obtain a second advertising material image with the style of the advertising text adjusted.

[0044] Preferably, the generating module is specifically used for: adjusting the size of the first advertising material picture; dividing the resized first advertising material picture into a plurality of picture blocks to form a corresponding picture column; adding a position information variable and a size information variable to the picture column to obtain a first sequence.

[0045] Preferably, the processing unit is specifically used for:

[0046] When the target advertisement category is a first advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement base image as a first advertisement material image related to the first advertisement category;

[0047] When the target advertisement category is a second advertisement category, inputting the prompt word to be processed into a preset image generation model to obtain an advertisement background image related to the second advertisement category;

[0048] The advertisement background image and the product image submitted by the user are merged and the image is illuminated to obtain a first advertisement material image.

[0049] Based on the above-mentioned embodiment of the present invention, a method and system for image processing are provided. The method comprises: obtaining prompt words and advertisement text to be processed related to a target advertisement category; inputting the prompt words to be processed into a preset image generation model to obtain a first advertisement material image related to the target advertisement category; calling a transformer and a diffusion model to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image. In this scheme, prompt words and advertisement text to be processed related to a target advertisement category are obtained. The image generation model is used to generate a first advertisement material image related to the target advertisement category based on the prompt words to be processed, and then the transformer and the diffusion model are used to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image. The advertisement material image can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by a designer, thereby improving the production efficiency of the advertisement material image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 A flowchart of a method for processing an image provided by an embodiment of the present invention;

[0052] Figure 2 A flow chart of obtaining a second advertising material picture provided by an embodiment of the present invention;

[0053] Figure 3 An example diagram of processing logic for obtaining text position and text size related to an advertisement text provided by an embodiment of the present invention;

[0054] Figure 4 An example diagram of processing logic for obtaining a second advertisement material image with the style of the advertisement text adjusted provided by an embodiment of the present invention;

[0055] Figure 5 Another flow chart of a picture processing method provided by an embodiment of the present invention;

[0056] Figure 6 A structural block diagram of an image processing system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0059] The generation of advertising materials can be divided into two parts: shooting the advertising base map and secondary editing. The current method of producing advertising materials is: for advertisements of different types, first arrange the scene and arrange the shooting of models according to the needs, then shoot the advertising base map, and finally the designer will perform secondary editing on the advertising base map to obtain the final advertising material image.

[0060] However, setting up the scene and shooting require a lot of manpower and material resources, and secondary editing of the advertising base map also takes time, resulting in low production efficiency and high production costs for advertising material images, and it is also impossible to mass-produce advertising material images.

[0061] To solve the above problems, this solution proposes a picture processing method and system to obtain prompt words and advertisement texts to be processed related to the target advertisement category. The picture generation model is used to generate a first advertisement material picture related to the target advertisement category based on the prompt words to be processed, and then the transformer and diffusion model are used to edit the first advertisement material picture based on the advertisement text to obtain a second advertisement material picture. The advertisement material picture can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by designers, thereby improving the production efficiency of advertisement material pictures and reducing production costs, and realizing batch production of advertisement material pictures.

[0062] See also Figure 1 , shows a flow chart of a picture processing method provided by an embodiment of the present invention, the method comprising:

[0063] Step S101: Obtain prompt words and advertisement texts to be processed that are related to the target advertisement category.

[0064] In the specific implementation of step S101, the advertisement description content and advertisement text related to the target advertisement category input by the user are obtained, and the advertisement text at least includes the advertisement title and / or advertisement selling point information; the advertisement description content is optimized using a prompt word optimization algorithm to obtain prompt words to be processed.

[0065] Among them, the target advertising category is the first advertising category or the second advertising category, the advertising background image related to the first advertising category can be used as the advertising material image, and the advertising background image related to the second advertising category needs to be integrated with the product image to obtain the advertising material image.

[0066] For example: the first advertising category can be red envelope or curiosity, etc., and the second advertising category can be e-commerce, etc.

[0067] In some embodiments, the prompt word optimization algorithm may be NeuroPrompts or other types of optimization algorithms, which can optimize the advertisement description content input by the user into prompt words that are more detailed and easier to be understood by the big model.

[0068] That is to say, the user pre-selects the target advertisement type for which the advertisement material image needs to be generated, and enters the advertisement description content (for example, a relatively concise description content); the prompt word optimization algorithm is used to optimize the advertisement description content entered by the user into a more detailed prompt word to be processed that is easier to be understood by the large model.

[0069] Step S102: inputting the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category.

[0070] It should be noted that the image generation model is trained based on sample data corresponding to multiple advertising categories and a fine-tuning model. The fine-tuning model is used to fine-tune the image generation model. The fine-tuning model can be a LoRA (Low-Rank Adaptation) model or other models that can be used to fine-tune a large model. The image generation model can be a Stable Diffusion model or other models that can be used to generate images.

[0071] To better understand how to train the image generation model, take the Stable Diffusion model as the image generation model and the LoRA model as the fine-tuning model as an example, the training process of the image generation model is explained through the following content:

[0072] Different LoRA models are used for different advertising categories. The LoRA model is added after the Stable Diffusion model to fine-tune the Stable Diffusion model. One LoRA model is used for each advertising category. For example, three LoRA models are used to fine-tune the Stable Diffusion model for red envelope, e-commerce and curiosity categories respectively.

[0073] When using the LoRA model corresponding to the red envelope class to fine-tune the Stable Diffusion model, a large number of sample data related to the red envelope class, such as limited-time red envelopes, full-discount coupons, and lucky draw wheels, are collected; these sample data related to the red envelope class are used in combination with the LoRA model corresponding to the red envelope class to fine-tune the Stable Diffusion model, so that the StableDiffusion model is suitable for generating advertising background images for the red envelope class (which can be used as advertising material images).

[0074] When using the LoRA model corresponding to e-commerce to fine-tune the Stable Diffusion model, let the Stable Diffusion model specifically learn the advertising background images of different products such as clothing, home appliances, furniture and food; after the Stable Diffusion model generates the advertising background image, the product image is proportionally reduced to an appropriate size (for example, reduced to 1 / 2 of the width or height of the advertising background image), and then the product image and the advertising background image are fused and illuminated, so that the product and background are more naturally integrated together, thereby obtaining e-commerce related advertising material images.

[0075] When using the LoRA model corresponding to the curiosity category to fine-tune the Stable Diffusion model, a large amount of sample data related to the curiosity category, such as surrealism and fantasy scenes, is collected. These sample data related to the curiosity category are combined with the LoRA model corresponding to the curiosity category to fine-tune the Stable Diffusion model, so that the Stable Diffusion model is suitable for generating curiosity category advertising background maps (which can be used as advertising material images).

[0076] Through the above method, the StableDiffusion model is fine-tuned using sample data and fine-tuning models corresponding to multiple advertising categories to obtain an image generation model.

[0077] The above is an explanation of how to train the image generation model.

[0078] In the specific implementation of step S102, the prompt word to be processed is input into the image generation model to obtain a first advertising material image related to the target advertising category; wherein, the method of obtaining the first advertising material image is different for different target advertising categories, which are explained below respectively:

[0079] (I) When the target advertisement category is the first advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement base image related to the first advertisement category as a first advertisement material image.

[0080] That is, when the target advertising category is the first advertising category (such as red envelope category or curiosity category), the prompt word to be processed is input into the image generation model to obtain the advertising base map related to the first advertising category output by the image generation model, and the advertising base map related to the first advertising category is used as the first advertising material image to be edited.

[0081] (ii) When the target advertisement category is the second advertisement category, a model is generated by inputting the image of the prompt word to be processed to obtain an advertisement background image related to the second advertisement category; the advertisement background image is merged and illuminated with the product image submitted by the user to obtain the first advertisement material image.

[0082] Specifically, when the target advertisement category is the second advertisement category, the image generation model is input with the prompt word to be processed, and an advertisement background image related to the second advertisement category generated by the image generation model is obtained, and the width and height of the advertisement background image are recorded as image_width and image_height respectively;

[0083] Load the product image submitted by the user and scale the size of the product image to a certain size, for example, scale the size of the product image to min(image_width, image_height) / 2;

[0084] The Poisson fusion algorithm is used to fuse the advertising background image and the proportionally scaled product image, and the IC-Light model is introduced to illuminate the fused image (the image obtained by fusion of the advertising background image and the proportionally scaled product image) to obtain the first advertising material image to be edited (which can more naturally fuse the product and background together). At this time, the first advertising material image is: the image obtained after lighting the fused image.

[0085] In summary, if the target advertisement category is the first advertisement category, the image generation model generates an advertisement base image, and the generated advertisement base image serves as the first advertisement material image to be edited;

[0086] If the target advertisement category is the second advertisement category, the image generation model generates an advertisement background image, and after fusing the advertisement background image with the product image and lighting the image, a first advertisement material image to be edited is obtained.

[0087] Step S103: calling the transformer and the diffusion model to edit the first advertisement material image based on the advertisement text to obtain a second advertisement material image.

[0088] In the specific implementation of step S103, after obtaining the first advertising material picture to be edited, the first advertising material picture is edited based on the advertisement text using transformer, diffusion model, Controlnet (conditional generative adversarial network) and unet (U-type network) to obtain the second advertising material picture.

[0089] In some embodiments, after obtaining the second advertising material image, a designated icon and a designated watermark are added to the second advertising material image; and the second advertising material image with the designated icon and the designated watermark added is delivered.

[0090] For example: after obtaining the second advertising creative image, add the icon (specified icon) to the second advertising creative image, and add the specified watermark "Generated by AI" to the lower right corner of the second advertising creative image. Thus, the advertising creative image that can be delivered is obtained (the second advertising creative image with the specified icon and the specified watermark added).

[0091] In the embodiment of the present invention, the prompt words to be processed and the advertisement text related to the target advertisement category are obtained. The image generation model is used to generate a first advertisement material picture related to the target advertisement category based on the prompt words to be processed, and then the transformer and the diffusion model are used to edit the first advertisement material picture based on the advertisement text to obtain a second advertisement material picture. The advertisement material picture can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by designers, thereby improving the production efficiency of the advertisement material pictures.

[0092] For the above-mentioned embodiments of the present invention Figure 1 The content of the second advertising material image obtained in step S103 is shown in FIG. Figure 2 , shows a flow chart of obtaining a second advertising material picture provided by an embodiment of the present invention, including the following steps:

[0093] Step S201: using the first advertisement material image, the position information variable and the size information variable to form a first sequence.

[0094] In the specific implementation process of step S201, the size of the first advertising material image is adjusted; the resized first advertising material image is divided into multiple image blocks to form a corresponding image column; a position information variable (denoted as P) and a size information variable (denoted as S) are added to the image column to obtain a first sequence (denoted as token0).

[0095] Specifically, the size of the first advertisement material image may be adjusted using the cv2.resize method. For example, the size of the first advertisement material image may be adjusted to 450*450 using the cv2.resize method.

[0096] Split the resized first creative image into multiple image blocks (each image block is denoted as img i ), to form a corresponding image column, for example: the first advertising creative image with a size of 450*450 is divided into 9 image blocks of 50*50 to form a corresponding image column {img1,…,img9}.

[0097] The first sequence (token0) is composed of a picture column, a position information variable (denoted as P), and a size information variable (denoted as S), token0={img1,…,img9,P,S}.

[0098] It should be noted that the position information variable is in the form of: P=[x,y,wrap]; where x represents the horizontal coordinate of the start of the text, y represents the vertical coordinate of the start of the text, and wrap is the line break information. If the maximum number of line breaks is 4, then wrap=[wrap1,wrap2, wrap3, wrap4]. iIndicates that the i-th line break occurs in wrap i After the word, if wrap i =0 means there is no i-th line break.

[0099] Step S202: performing word segmentation processing on the advertisement text to obtain a second sequence.

[0100] In the specific implementation of step S202 , the advertisement text is segmented to obtain a second sequence (denoted as token1 ). For example, the advertisement text is tokenized at a word granularity to form a second sequence token1 including a plurality of vocabulary units.

[0101] Step S203: input the first sequence and the second sequence into a transformer for encoding and decoding to obtain a text position and a text size related to the advertisement text.

[0102] Input the first sequence (token0) and the second sequence (token1) into the transformer for encoding (encoder) and decoding (decoder) to obtain the text position and text size related to the advertisement text;

[0103] It should be noted that the transformer contains encoder and decoder;

[0104] The first sequence (token0) and the second sequence (token1) are input into the transformer. After the encoder and decoder, the text position and text size related to the advertisement text (ad title and / or advertisement selling point information) can be obtained. The obtained text position is recorded as P1, and the obtained text size is recorded as S1.

[0105] For example Figure 3 The example diagram of the processing logic for obtaining the text position and text size related to the advertisement text can be seen in the figure. Figure 3 The first ad creative image in the advertisement is the advertisement background image, and the advertisement text includes the advertisement title and advertisement selling point information;

[0106] Change the first creative image ( Figure 3The advertising base image in the image is tokenized to divide the first advertising material image into an image column containing multiple image blocks, and then the image column, position information variable (Position) and size information variable (Size) are input into the transformer, where Embedding is represented as serialization; the advertising text is tokenized at word granularity and input into the transformer; after the encoder and decoder, the text position (Position, i.e. the P1 mentioned above) and text size (Size, i.e. the S1 mentioned above) related to the advertising text can be obtained.

[0107] Step S204: Add the advertisement text having a size equal to the text size to the text position of the first advertisement material picture to obtain a first intermediate picture.

[0108] In the specific implementation of step S204 , after obtaining the text position ( P1 ) and text size ( S1 ) of the advertisement text, the advertisement text having a size of the text size S1 is added to the text position P1 of the first advertisement material image to obtain the first intermediate image.

[0109] That is to say, the advertisement text with a size of S1 is added to the P1 of the first advertisement material picture (equivalent to pasting words in the first advertisement material picture) to obtain the first intermediate picture.

[0110] In summary, the first middle image is: the first advertising material image with the advertising text of size S1 added at P1.

[0111] Step S205: calling the diffusion model and Controlnet to extract edge information of the target object and the advertisement text in the first intermediate image to obtain a second intermediate image.

[0112] In the specific implementation process of step S205, after obtaining the first intermediate image, the diffusion model and Controlnet are called to extract the edge information of the target object (such as the main character) and the advertising text in the first intermediate image to obtain the second intermediate image. The extracted edge information of the target object and the advertising text can ensure that the target object and the advertising text are not deformed.

[0113] That is to say, the first intermediate image can obtain edge information of the target object and the advertisement text in the first intermediate image through the diffusion model and Controlnet, thereby obtaining the second intermediate image.

[0114] Step S206: Create a mask according to the text position.

[0115] In the specific process of implementing step S206 , a mask is created according to the text position P1 of the advertisement text.

[0116] It should be noted that the specific method of making the mask is: according to the text position P1 (including coordinates and line break information) and text size S1 of the advertisement text, the text area occupied by the advertisement text can be determined, and the text area is set to white and other areas are set to black, thereby making a mask.

[0117] Step S207: input the mask and the second intermediate image into unet to obtain a second advertisement material image with the style of the advertisement text adjusted.

[0118] In the specific implementation of step S207, the mask and the second intermediate image are input into unet to regenerate the text area, thereby obtaining a second advertising material image with the style of the advertising text adjusted;

[0119] Compared to the original and unadjusted ad text, the style of the ad text in the second ad creative image is adjusted;

[0120] For example, the font of the advertisement text in the second advertisement creative image is adjusted in any combination of the following ways: making the font bold, making a part of the font round, adding a shadow, and adding a gradient color.

[0121] Another example Figure 4 As shown in the example diagram of the processing logic for obtaining the second advertising material image with the style of the advertising text adjusted, Figure 4 The first ad creative image in the advertisement is the advertisement background image, and the advertisement text includes the advertisement title and advertisement selling point information;

[0122] The text position of the advertisement text determined as above ( Figure 4 Position in ) and text size ( Figure 4 ), add an advertisement text having a size of S1 at the text position P1 of the first advertisement material picture to obtain a first intermediate picture;

[0123] The first intermediate image is processed through a variational autoencoder (VAE), a diffusion model, and a Controlnet to extract edge information of the target object and the advertisement text in the first intermediate image to obtain a second intermediate image, which is a black background image.

[0124] A mask is created according to the text position of the advertisement text; the mask and the second intermediate image are passed through unet and VAE to obtain a second advertisement material image with the style of the advertisement text adjusted.

[0125] The above embodiments of the present invention Figure 2, is an explanation of editing a first advertising material image to obtain a second advertising material image.

[0126] To further help understand this solution, the following Figure 5 , this scheme is illustrated by examples from the practical application aspect; see Figure 5 , shows another flow chart of a picture processing method provided by an embodiment of the present invention, comprising the following steps:

[0127] Step S501: Acquire the advertisement description content, advertisement title and advertisement selling point information related to the target advertisement category input by the user.

[0128] It should be noted that the advertisement title and advertisement selling point information are the advertisement text.

[0129] Step S502: Prompt optimization is performed on the advertisement description content to obtain prompt words to be processed.

[0130] Step S503: inputting the prompt word to be processed into the image generation model to obtain a first advertising material image related to the target advertising category.

[0131] In the specific implementation of step S503, the target advertisement category is selected; if the target advertisement category is red envelope or curiosity, the prompt word to be processed is input into the image generation model to obtain the advertisement base map output by the image generation model, and the advertisement base map is used as the first advertisement material image to be edited.

[0132] If the target advertisement category is e-commerce, the image generation model with the prompt word to be processed is input, and the advertisement background image output by the image generation model is obtained; at this time, the product image is obtained, and the size of the product image is proportionally scaled to min(image_width, image_height) / 2; the advertisement background image and the proportionally scaled product image are merged, and the IC-Light model is introduced for image lighting to obtain the first advertisement material image to be edited.

[0133] Step S504: Calculate the text position and text size of the advertisement title and advertisement selling point information.

[0134] Step S505: Based on the text position and text size, the advertisement title and advertisement selling point information are added to the first advertisement material picture and edited to obtain a second advertisement material picture with the style of the advertisement text adjusted.

[0135] Step S506: adding a designated watermark to the second advertising material image and delivering the advertisement.

[0136] It should be noted that the execution principle of steps S501 to S506 can be found in the above-mentioned embodiment of the present invention. Figure 1and Figure 2 The content will not be repeated here.

[0137] From the contents of the above embodiments, it can be seen that the present solution utilizes an image generation model to generate a first advertising material image, and performs secondary editing on the first advertising material image to obtain a second advertising material image. The advertising material image can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by a designer, and can realize batch production of advertising material images, thereby improving the production efficiency of advertising material images and reducing production costs.

[0138] Corresponding to the image processing method provided by the above embodiment of the present invention, see Figure 6 , the embodiment of the present invention also provides a structural block diagram of a picture processing system, the system includes: an acquisition unit 100, a processing unit 200 and an editing unit 300;

[0139] The acquisition unit 100 is used to acquire prompt words and advertisement texts to be processed that are related to the target advertisement category.

[0140] In some embodiments, the acquisition unit 100 is specifically used to: obtain the advertisement description content and advertisement text related to the target advertisement category input by the user, the advertisement text at least including the advertisement title and / or advertisement selling point information; optimize the advertisement description content using a prompt word optimization algorithm to obtain the prompt word to be processed.

[0141] The processing unit 200 is used to input the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category, wherein the image generation model is trained based on sample data corresponding to multiple advertising categories and a fine-tuning model.

[0142] In some embodiments, the processing unit 200 is specifically used to: when the target advertisement category is the first advertisement category, input the prompt word to be processed into a preset image generation model to obtain an advertisement base image related to the first advertisement category as a first advertisement material image;

[0143] When the target advertisement category is the second advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement background image related to the second advertisement category;

[0144] The advertisement background image and the product image submitted by the user are merged and the image is illuminated to obtain the first advertisement material image.

[0145] The editing unit 300 is used to call the transformer and the diffusion model to edit the first advertisement material picture based on the advertisement text to obtain the second advertisement material picture.

[0146] In the embodiment of the present invention, the prompt words to be processed and the advertisement text related to the target advertisement category are obtained. The image generation model is used to generate a first advertisement material picture related to the target advertisement category based on the prompt words to be processed, and then the transformer and the diffusion model are used to edit the first advertisement material picture based on the advertisement text to obtain a second advertisement material picture. The advertisement material picture can be automatically generated without the need for scene arrangement, shooting, and manual secondary editing by designers, thereby improving the production efficiency of the advertisement material pictures.

[0147] Preferably, combined Figure 6 The content shown, the editing unit 300 includes a generation module, a word segmentation module, a processing module, an adding module, an extraction module, a production module and an adjustment module, and the execution principle of each module is as follows:

[0148] The generating module is used to form a first sequence by using the first advertising material image, the position information variable and the size information variable.

[0149] In some embodiments, the generation module is specifically used to: adjust the size of the first advertising material image; divide the resized first advertising material image into multiple image blocks to form a corresponding image column; add position information variables and size information variables to the image column to obtain a first sequence.

[0150] The word segmentation module is used to perform word segmentation processing on the advertisement text to obtain a second sequence.

[0151] The processing module is used to input the first sequence and the second sequence into the transformer for encoding and decoding to obtain the text position and text size related to the advertisement text.

[0152] The adding module is used to add the advertisement text with the size of the text size to the text position of the first advertisement material picture to obtain the first intermediate picture.

[0153] The extraction module is used to call the diffusion model and Controlnet to extract the edge information of the target object and the advertisement text in the first intermediate image to obtain the second intermediate image.

[0154] The production module is used to produce masks according to the text position.

[0155] The adjustment module is used to input the mask and the second intermediate image into unet to obtain a second advertising material image with the style of the advertising text adjusted.

[0156] Preferably, combined Figure 6 The system further comprises:

[0157] An adding unit, used for adding a specified icon and a specified watermark to the second advertising creative image.

[0158] The delivery unit is used to deliver the second advertising creative image with the specified icon and the specified watermark added.

[0159] In summary, the embodiment of the present invention provides an image processing method and system to obtain prompt words to be processed and advertisement texts related to a target advertisement category. A first advertisement material image related to the target advertisement category is generated based on the prompt words to be processed using an image generation model, and then the first advertisement material image is edited based on the advertisement text using a transformer and a diffusion model to obtain a second advertisement material image. The advertisement material image can be automatically generated without the need for scene arrangement, shooting, or manual secondary editing by a designer, thereby improving the production efficiency of the advertisement material image.

[0160] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0161] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0162] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing an image, characterized in that: The method comprises: Obtain pending prompt words and advertisement texts related to the target advertisement category; Inputting the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category, wherein the image generation model is trained based on sample data corresponding to multiple advertising categories and a fine-tuning model; Calling transformer and diffusion model to edit the first advertising material image based on the advertising text to obtain the second advertising material image, including: using the first advertising material image, position information variables and size information variables to form a first sequence; performing word segmentation on the advertising text to obtain a second sequence; inputting the first sequence and the second sequence into transformer for encoding and decoding to obtain text position and text size related to the advertising text; adding the advertising text with a size of the text size to the text position of the first advertising material image to obtain a first intermediate image; calling diffusion model and Controlnet to extract the target object in the first intermediate image and the edge information of the advertising text to obtain a second intermediate image; making a mask according to the text position; inputting the mask and the second intermediate image into unet to obtain the second advertising material image with the style of the advertising text adjusted.

2. The method according to claim 1, characterized in that The first advertising material image, the position information variable and the size information variable are used to form a first sequence, including: Adjusting the size of the first advertising creative image; Splitting the resized first advertising creative image into a plurality of image blocks to form corresponding image columns; A position information variable and a size information variable are added to the picture sequence to obtain a first sequence.

3. The method according to claim 1, characterized in that Inputting the prompt word to be processed into a preset picture generation model to obtain a first advertising material picture related to the target advertising category, including: When the target advertisement category is a first advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement base image as a first advertisement material image related to the first advertisement category; When the target advertisement category is a second advertisement category, inputting the prompt word to be processed into a preset image generation model to obtain an advertisement background image related to the second advertisement category; The advertisement background image and the product image submitted by the user are merged and the image is illuminated to obtain a first advertisement material image.

4. The method according to claim 1, characterized in that The step of obtaining the prompt words and advertisement text to be processed related to the target advertisement category includes: Acquire the advertisement description content and advertisement text related to the target advertisement category input by the user, wherein the advertisement text at least includes the advertisement title and / or advertisement selling point information; The advertisement description content is optimized using a prompt word optimization algorithm to obtain prompt words to be processed.

5. The method according to any one of claims 1 to 4, characterized in that: After obtaining the second advertising material image, it also includes: Adding a specified icon and a specified watermark to the second advertising creative image; The second advertising creative image with the specified icon and the specified watermark added is delivered.

6. A picture processing system, characterized in that: The system comprises: An acquisition unit, used for acquiring prompt words and advertisement texts to be processed that are related to the target advertisement category; A processing unit, configured to input the prompt word to be processed into a preset image generation model to obtain a first advertising material image related to the target advertising category, wherein the image generation model is trained based on sample data corresponding to a plurality of advertising categories and a fine-tuning model; An editing unit, configured to call a transformer and a diffusion model to edit the first advertising material image based on the advertising text to obtain a second advertising material image; The editing unit includes: a generation module, a word segmentation module, a processing module, an adding module, an extraction module, a production module and an adjustment module; The generating module is used to form a first sequence using the first advertising material picture, the position information variable and the size information variable; The word segmentation module is used to perform word segmentation processing on the advertisement text to obtain a second sequence; The processing module is used to input the first sequence and the second sequence into a transformer for encoding and decoding, so as to obtain a text position and a text size related to the advertisement text; The adding module is used to add the advertisement text having a size equal to the text size to the text position of the first advertisement material picture to obtain a first intermediate picture; The extraction module is used to call the diffusion model and Controlnet to extract the edge information of the target object and the advertisement text in the first intermediate image to obtain the second intermediate image; The making module is used to make a mask according to the text position; The adjustment module is used to input the mask and the second intermediate image into unet to obtain a second advertising material image with the style of the advertising text adjusted.

7. The system according to claim 6, characterized in that The generation module is specifically used to: adjust the size of the first advertising material image; divide the resized first advertising material image into multiple image blocks to form a corresponding image column; add position information variables and size information variables to the image column to obtain a first sequence.

8. The system according to claim 6, characterized in that The processing unit is specifically used for: When the target advertisement category is a first advertisement category, the prompt word to be processed is input into a preset image generation model to obtain an advertisement base image as a first advertisement material image related to the first advertisement category; When the target advertisement category is a second advertisement category, inputting the prompt word to be processed into a preset image generation model to obtain an advertisement background image related to the second advertisement category; The advertisement background image and the product image submitted by the user are merged and the image is illuminated to obtain a first advertisement material image.

Images