Poster picture generation method and device based on artificial intelligence, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a poster picture generation method and device based on artificial intelligence, equipment and a medium, and the method comprises the following steps: generating a text masking picture according to text content information of a selected poster template picture and a target poster picture, the text content information comprises main title information, subtitle information and text information; according to a user prompt word, the stable diffusion model is utilized to generate an intermediate picture corresponding to the user prompt word, and the user prompt word comprises text description information of the target poster picture; according to the user prompt words and the character masking map, the stable diffusion model is utilized to adjust the intermediate picture, and a poster base map of the target poster picture is generated; and adding the text content information into the poster base picture to generate the target poster picture. The technical problems of low poster picture making efficiency and high cost in the existing scheme are solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based poster image generation method, device, equipment and medium. Background Art

[0002] With the development of the Internet and the popularization of the network, the speed of information dissemination is getting faster and faster. Enterprises use various methods to advertise on the Internet to promote their products. Among them, posters are an important promotional medium. In the future advertising field, precision marketing is a major trend. Under the background of large traffic, posters need to show the effect of "thousands of faces for thousands of people", which requires the rapid generation of a large number of personalized posters.

[0003] The existing manual poster production method is inefficient, costly, and has a low data reuse rate. However, as the number of platform users increases, more poster images are needed. The manual poster production method is difficult to meet the real-time and large-scale poster image needs. Summary of the invention

[0004] The present invention provides a poster image generation method, device, computer equipment and medium based on artificial intelligence to solve the technical problems of low efficiency and high cost in poster image production in existing solutions.

[0005] In a first aspect, a poster image generation method based on artificial intelligence is provided, comprising:

[0006] Generate a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0007] According to the user prompt, using the stable diffusion model, generating an intermediate image corresponding to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0008] According to the user prompt and the text mask image, the intermediate image is adjusted using the stable diffusion model to generate a poster base image of the target poster image;

[0009] The text content information is added to the poster base image to generate the target poster image.

[0010] In a second aspect, a poster image generation device based on artificial intelligence is provided, comprising:

[0011] A mask module, used to generate a text mask image according to text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0012] A synthesis module, configured to generate an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0013] A base map module, used to adjust the intermediate image according to the user prompt and the text mask image and to generate a poster base map of the target poster image by using the stable diffusion model;

[0014] A generation module is used to add the text content information to the poster base image to generate the target poster image.

[0015] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned artificial intelligence-based poster image generation method when executing the computer program.

[0016] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned artificial intelligence-based poster image generation method are implemented.

[0017] In the scheme implemented by the above-mentioned artificial intelligence-based poster image generation method, device, computer equipment and storage medium, when it is necessary to generate a target poster image, a poster template image is selected, and text content information is entered to generate a text mask image; and based on the user prompt input by the user for text description of the target poster image, an intermediate image is generated using a stable diffusion model; then the intermediate image is adjusted using the text mask image and the stable diffusion model to generate a poster base image; finally, the text content information is added to the poster base image to obtain the target poster image.

[0018] In the embodiment of the present invention, to generate a target poster image, it is only necessary to select a poster template image, input text content information and user prompts, and then the target poster image can be generated. There is no need to introduce additional manpower or models for typesetting, which reduces resource consumption, shortens the time for generating the target poster image, and can generate poster images in batches. Furthermore, there is no need to introduce an additional control model to import poster layout information for the stable diffusion model, which reduces resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 It is a schematic diagram of an application environment of a poster image generation method based on artificial intelligence in one embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a poster image generation method based on artificial intelligence in one embodiment of the present invention;

[0022] Figure 3 for Figure 2 A schematic flow chart of a specific implementation of step S10;

[0023] Figure 4 A schematic diagram of a binary poster image provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of a text mask provided by an embodiment of the present invention;

[0025] Figure 6 for Figure 2 A schematic flow chart of a specific implementation of step S20;

[0026] Figure 7 A method for performing T is provided in an embodiment of the present invention. 1 Schematic diagram of the denoising operation;

[0027] Figure 8 for Figure 2 A schematic flow chart of a specific implementation of step S30;

[0028] Fig. 9 A method for performing T is provided in an embodiment of the present invention. 2 Schematic diagram of the denoising operation;

[0029] Fig.10 for Figure 2 A schematic flow chart of a specific implementation of step S40;

[0030] Fig.11 It is a structural schematic diagram of a poster image generating device based on artificial intelligence in one embodiment of the present invention;

[0031] Fig.12 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention;

[0032] Fig.13 A schematic diagram of the structure of a computer device provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The poster image generation method based on artificial intelligence provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the client communicates with the server through a network. The server can receive text content information and user prompts through the client, and generate a text mask map according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes the main title information, the subtitle information and the text information; according to the user prompt, using the stable diffusion model, generate an intermediate image corresponding to the user prompt, and the user prompt includes the text description information of the target poster image; according to the user prompt and the text mask map, using the stable diffusion model, adjust the intermediate image to generate the poster base map of the target poster image; add the text content information to the poster base map to generate the target poster image, and feed the target poster image back to the client. In an embodiment of the present invention, when it is necessary to generate the target poster image, select the poster template image, input the text content information, generate the text mask map; and according to the user prompt input by the user for the text description of the target poster image, use the stable diffusion model to generate the intermediate image; then use the text mask map and the stable diffusion model to adjust the intermediate image to generate the poster base map; finally, add the text content information to the poster base map to obtain the target poster image.

[0035] In the embodiment of the present invention, to generate a target poster image, it is only necessary to select a poster template image, input text content information and user prompts, and then the target poster image can be generated. No additional manpower or model is required for typesetting, which reduces resource consumption, shortens the time for generating the target poster image, and can generate poster images in batches; and, no additional control model is required to import poster layout information for the stable diffusion model, which reduces resource consumption. The present invention is described in detail below through specific embodiments.

[0036] A flowchart of a poster image generation method based on artificial intelligence provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the method comprises the following steps:

[0037] S10: generating a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0038] In the embodiment of the present invention, a text mask image is generated according to the text content information of the selected poster template image and the target poster image. The poster template image may be selected from a poster template library, or may be obtained by analyzing other poster images using an OCR (Optical Character Recognition) model, a multimodal model, etc. In the embodiment of the present invention, selection from a poster template library is used as an example for explanation.

[0039] Specifically, the poster template library includes a plurality of poster template images of different styles, and the poster template images include three parts: poster image size information, poster content information, and poster layout information. The poster image size information includes the width and height of the poster image; the poster content information includes the main title, subtitle, and text, as well as the font, font size, line height, text art effect, and other information of the three types of poster content, wherein the text art effect includes faded text, filled font, split font, hollow text, shadow, etc., and the specific types can be determined according to the actual situation, and the embodiment of the present invention does not make specific restrictions on this; the poster layout information includes the placement area of ​​the poster content information and the typesetting style of the poster content, wherein the placement area is the area in the poster image where the text content is placed. For example, there is a picture in a certain area of ​​some poster images. If text is placed in this area, the picture will be blocked, thereby affecting the advertising effect. Therefore, the placement area is a reasonable position area for placing text in the poster image. The placement area in each poster template image is set in advance and is determined according to the actual situation. The embodiment of the present invention does not make specific restrictions on this.

[0040] After the poster template library is established, a poster template image is selected from the poster template library according to the style of the target poster image, and a text mask image is generated based on the poster template image and the text content information input by the user. The text content information input by the user is the text content in the target poster image. The user can determine the text content according to the target poster image he wants. For example, in a poster image describing the Spring Festival, the main title is "Happy Spring Festival", the subtitle is "Reunion, Joy, Peace", and the main text information is "China's most important traditional festival attracts hundreds of millions of people to go home and reunite with their families every year."

[0041] As an optional implementation, Figure 3 As shown, the steps of generating a text mask image according to the text content information of the selected poster template image and the target poster image include:

[0042] S11: setting the background area in the poster template image to 0 and the text content to 1 to obtain a binary poster image;

[0043] After selecting a poster template image, the background area in the poster template image is set to 0, and the poster text content is set to 1 to obtain a binary poster image.

[0044] An embodiment of the present invention provides a binary poster image, such as Figure 4 As shown, in the embodiment of the present invention, except for the text in the poster template image, the rest can be regarded as the background area, and the chromaticity value of the background area is set to 0, and the chromaticity value of the text content is set to 1, that is, the background area is black and the text is white. Figure 4 In the binary poster image shown, "Happy Chinese New Year", "Reunion, Joy and Peace" and "China's most important traditional festival attracts hundreds of millions of people to go home and reunite with their families every year" have no text content and are represented in white; the background behind is represented in black.

[0045] S12: Determine the minimum inscribed rectangle of the text content in the binary poster image, and set the value of the minimum inscribed rectangle to 1;

[0046] Please continue reading Figure 5 , determine the minimum inscribed rectangle of “Happy Chinese New Year”, the minimum inscribed rectangle of “reunion, joy and peace” and the minimum inscribed rectangle of “China’s most important traditional festival attracts hundreds of millions of people to go home and reunite with their families every year”.

[0047] S13: Using the minimum inscribed rectangle to cover the text content, to obtain the text mask image.

[0048] Set the chromaticity value of the area where the minimum inscribed rectangle of each text content is located to 1, and finally obtain a text mask image. Please continue to refer to Figure 5 "Happy Chinese New Year", "Reunion, Joy, Harmony", "China's most important traditional festival attracts hundreds of millions of people to go home and reunite with their families every year" are all covered by white rectangular areas.

[0049] S20: generating an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0050] The user prompt is input into the stable diffusion model to generate an intermediate image, wherein the user prompt is generally a descriptive text input by the user to describe the content expressed by the target poster image. For example, the user prompt may be "During the Spring Festival, the night is full of stars, the moon is bright, and astronauts are also exploring the blue outer space." The intermediate image is generated by the stable diffusion model according to the user prompt and is used to describe the image represented by the user prompt.

[0051] Among them, the Stable Diffusion model is a Stable Diffusion model. In the field of image processing, image generation technology has always been a research hotspot, and the Stable Diffusion model has become the most popular image generation model in the field of image generation. Through the Stable Diffusion model, even ordinary users who have not received professional art education can generate high-quality, detailed, and colorful pictures. Moreover, the model can be run on civilian-grade graphics cards and can generate a picture in just a few seconds. Therefore, it is very suitable for automatically and quickly generating a large number of personalized poster picture base maps to achieve the effect of "thousands of faces for thousands of people".

[0052] However, since this generation method does not refer to the poster layout information, the generated poster base map can only be obtained through manual design or automatic design using an additional typesetting model to obtain the finished poster image. Although the manual design method has a high yield rate and good visual effects, it is inefficient and costly, which offsets the advantages of the Stable Diffusion model in automatically generating images; although the method using an additional typesetting model can automatically perform typesetting and layout, it will introduce new computing resource consumption, and if there are too many elements in the poster base map and the picture is too complex, the image cannot be directly typeset to obtain the finished poster image, which reduces the yield rate of the poster image.

[0053] In the embodiment of the present invention, poster typesetting layout information is added to the poster template image, so that the generated poster base image can naturally adapt to the poster layout, thereby improving the output yield of the poster image.

[0054] As an optional implementation, Figure 6 As shown, the steps of generating an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt include:

[0055] S21: inputting the user prompt into the text encoder of the stable diffusion model to obtain text features;

[0056] The user prompt is input into the text encoder of the stable diffusion model to obtain text features. The text encoder is the Text Encoder model in the Stable Diffusion model.

[0057] S22: According to the text features, the initialization noise features are T 1 The first denoising operation is performed to obtain the first final noise feature, where T 1 is a positive integer;

[0058] Then, the initialization noise feature is sampled from the standard multi-dimensional Gaussian distribution, and the size of the initialization noise feature is 4×H×W, where H is the width of the text mask image and is evenly divisible by 8, and W is the height of the text mask image and is evenly divisible by 8.

[0059] Combined with text features, the initial noise features are T 1 The first denoising operation is performed to obtain the first final noise feature, where T 1 is a positive integer, and the specific value can be determined according to actual conditions, and the embodiment of the present invention does not make any specific limitation on this.

[0060] Among them, t1 is the starting point of the text-to-image time step, corresponding to the first denoising operation; t1+ΔT1 is the time step of the second denoising operation, and so on, t1+(T 1 -1)×ΔT1 is the T 1 The time step corresponding to the denoising operation. Both t1 and ΔT1 can be determined according to actual conditions, and the embodiment of the present invention does not make any specific limitation on this.

[0061] As an example, a denoising operation schematic diagram provided by an embodiment of the present invention is as follows: Figure 7 As shown, the initialization noise feature is T based on the text feature. 1 A denoising operation is performed to obtain a first final noise feature, the steps comprising:

[0062] For the Sth denoising operation, the predicted noise corresponding to the S-1th time step and the text feature are input into the U-type network of the stable diffusion model to obtain the predicted noise corresponding to the Sth time step, where 1≤S≤T 1 , when S=1, the predicted noise corresponding to the S-1th time step is the initialization noise feature;

[0063] Repeat the above steps T 1 The predicted noise output by the U-shaped network of the stable diffusion model is used as the first final noise feature.

[0064] S23: Inputting the first final noise feature into the decoder of the variational autoencoder of the stable diffusion model to obtain the intermediate image.

[0065] Finally, the first final noise feature is input into the decoder of the variational autoencoder of the stable diffusion model, that is, the first final noise feature is input into the Decoder of the VAE model in the Stable Diffusion model, and the intermediate picture is output.

[0066] Among them, the Variational Autoencoder (VAE) is a generative model that combines the ideas of autoencoders and variational inference, and is mainly used for tasks such as data generation, dimensionality reduction, and feature learning.

[0067] S30: adjusting the intermediate image according to the user prompt and the text mask image using the stable diffusion model to generate a poster base image of the target poster image;

[0068] Then the user prompt and the text mask are input into the stable diffusion model to adjust the intermediate image, and the stable diffusion model can output the poster base image of the target poster image.

[0069] It is easy to understand that the middle picture is a picture generated according to the user prompt, and there is no connection between it and the text mask picture. Therefore, it is also necessary to use the text mask picture to adjust the middle picture so that the poster base picture better meets user needs.

[0070] As an optional implementation, Figure 8 As shown, in step S30, the intermediate image is adjusted according to the user prompt and the text mask image by using the stable diffusion model to generate a poster base image of the target poster image, and the steps include:

[0071] S31: inputting the user prompt into the text encoder of the stable diffusion model to obtain text features;

[0072] Input the user prompts into the Text Encoder model in the Stable Diffusion model to obtain text features. The specific steps are the same as the above process. Please refer to the above steps for details.

[0073] S32: inputting the preprocessed intermediate image into the encoder of the variational autoencoder of the stable diffusion model to obtain image features;

[0074] The intermediate image is preprocessed, including mosaic processing and Gaussian blur processing. Then the preprocessed intermediate image is input into the encoder of the variational autoencoder of the stable diffusion model to obtain the image features. The encoder of the variational autoencoder is the Encoder model (i.e., encoder) of the VAE model.

[0075] S33: Perform T on the image features 2 The noise addition operation is performed to obtain the initial image features, where T 2 is a positive integer;

[0076] T 2Noise operation is performed to obtain the initial image features, T 2 is a positive integer, and the specific value can be determined according to actual conditions, and the embodiment of the present invention does not make any specific limitation on this.

[0077] S34: Based on the text features and the text mask, using the stable diffusion model, perform T on the initialization image features. 2 A denoising operation is performed to obtain a second final noise feature;

[0078] Then proceed to T 2 The denoising step is as follows: Fig. 9 As shown, for the Qth image-to-image time step, the text features and the image noise features corresponding to the Q-1th image-to-image time step are input into the U-type network to obtain the second prediction noise corresponding to the Qth image-to-image time step; and the second prediction noise corresponding to the Qth image-to-image time step is denoised by a desampler to obtain the second prediction noise corresponding to the Q-1th image-to-image time step; the image noise features corresponding to the Q-1th image-to-image time step are obtained by using the second prediction noise corresponding to the Q-1th image-to-image time step and the image features corresponding to the Q-1th image-to-image time step.

[0079] Among them, when Q=1, the image noise feature corresponding to the Q-1th image-to-image time step is the initialization image feature.

[0080] Furthermore, the image noise feature corresponding to the Q-1th image-to-image time step is obtained by using the second prediction noise corresponding to the Q-1th image-to-image time step and the image feature corresponding to the Q-1th image-to-image time step. The calculation formula is as follows:

[0081] X q-1 =X q ' -1 Θ(1-M)+i q-1 ΘM;

[0082] X q-1 is the image noise feature corresponding to the Q-1th image-to-image time step, X q ' -1 is the image feature corresponding to the Q-1th image-to-image time step, M is the text feature, i q-1 It is the second prediction noise corresponding to the Q-1th graph-to-graph time step.

[0083] It should be noted that t2 is the starting time step of the image-to-image conversion in the above denoising process, corresponding to the first denoising operation; t2+ΔT2 is the time step of the second denoising operation, and so on. 2 -1)×ΔT2 is the T 1The time step corresponding to the denoising operation. Both t2 and ΔT2 can be determined according to actual conditions, and the embodiment of the present invention does not make any specific limitation on this.

[0084] S35: Input the second final noise feature into the decoder of the variational autoencoder of the stable diffusion model to obtain the poster base image.

[0085] The second final noise feature is input into the decoder of the variational autoencoder of the temperature diffusion model to obtain the poster base image. The decoder of the variational autoencoder is the Decoder model (i.e., decoder) of the VAE model.

[0086] S40: Add the text content information to the poster base image to generate the target poster image.

[0087] Finally, the text content information is added to the poster base image to obtain the target poster image.

[0088] As an example, Fig.10 As shown, the poster template image includes a placement area for text content, and the steps of adding the text content information to the poster base image to generate the target poster image include:

[0089] S41: extracting colors whose area occupies more than a preset proportion in the poster base image as candidate colors, wherein the candidate colors include white and black;

[0090] S42: determining a final text color of the placement area according to the placement area and the candidate colors;

[0091] S43: generating the target poster image according to the text content information, the placement area of ​​the text content, and the final text color of the placement area.

[0092] As an example, the step of determining the final text color of the placement area according to the placement area and the candidate colors includes:

[0093] Calculating the contrast between the tile color of the placement area and each candidate color by a perceptual contrast algorithm;

[0094] The alternative color with a contrast ratio greater than the preset contrast ratio is used as the target text color;

[0095] The final text color is determined according to the target text color.

[0096] In the embodiment of the present invention, the candidate colors whose area ratio in the poster base image exceeds the preset ratio are extracted and form a candidate color group together with black and white, wherein the preset ratio can be determined according to actual conditions and is not specifically limited in the embodiment of the present invention. For example, the preset ratio can be 5%.

[0097] Then, based on the placement area and poster template image given by the user, the position and size of each poster content area can be determined, the placement area of ​​the poster content is traversed, the block is obtained in the area of ​​the corresponding position and size in the poster base map, and the main color of the block is calculated. Then, each alternative color in the alternative color group is traversed, and the absolute value of the APCA contrast ratio between each alternative color and the main color of the block is calculated.

[0098] If the absolute value is greater than the preset contrast, the specific value of the preset contrast can be determined according to the actual situation, and the embodiment of the present invention does not specifically limit this. For example, in the embodiment of the present invention, the preset contrast is 40, that is, when the absolute value is greater than 40, the candidate color is added to the qualified color group. After the candidate color group is traversed, a color is randomly selected from the qualified color group as the font color of the poster content. Finally, according to the specific poster content and the corresponding font color, poster template and poster background, the final poster image is rendered.

[0099] In the scheme implemented by the above-mentioned artificial intelligence-based poster image generation method, when it is necessary to generate a target poster image, a poster template image is selected, and text content information is input to generate a text mask image; and based on the user prompt input by the user for text description of the target poster image, an intermediate image is generated using a stable diffusion model; then the intermediate image is adjusted using the text mask image and the stable diffusion model to generate a poster base image; finally, the text content information is added to the poster base image to obtain the target poster image.

[0100] In the embodiment of the present invention, to generate a target poster image, it is only necessary to select a poster template image, input text content information and user prompts, and then the target poster image can be generated. There is no need to introduce additional manpower or models for typesetting, which reduces resource consumption, shortens the time for generating the target poster image, and can generate poster images in batches. Furthermore, there is no need to introduce an additional control model to import poster layout information for the stable diffusion model, which reduces resource consumption.

[0101] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0102] In one embodiment, a poster image generation device based on artificial intelligence is provided, and the poster image generation device based on artificial intelligence corresponds to the poster image generation method based on artificial intelligence in the above embodiment. Fig.11 As shown, the poster image generation device based on artificial intelligence includes a mask module 111, a synthesis module 112, a base map module 113 and a generation module 114. The detailed description of each functional module is as follows:

[0103] The mask module 111 is used to generate a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0104] A synthesis module 112, configured to generate an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0105] A base map module 113 is used to adjust the intermediate image according to the user prompt and the text mask image by using the stable diffusion model to generate a poster base map of the target poster image;

[0106] The generating module 114 is used to add the text content information to the poster base image to generate the target poster image.

[0107] In one embodiment, the synthesis module 112 is specifically configured to:

[0108] Inputting the user prompt into the text encoder of the stable diffusion model to obtain text features;

[0109] According to the text features, the initialization noise features are T 1 The first denoising operation is performed to obtain the first final noise feature, where T 1 is a positive integer;

[0110] The first final noise feature is input into a decoder of the variational autoencoder of the stable diffusion model to obtain the intermediate image.

[0111] In one embodiment, the base map module 113 is specifically used for:

[0112] Inputting the user prompt into the text encoder of the stable diffusion model to obtain text features;

[0113] Inputting the preprocessed intermediate image into the encoder of the variational autoencoder of the stable diffusion model to obtain image features;

[0114] The image features are T2 The noise addition operation is performed to obtain the initial image features, where T 2 is a positive integer;

[0115] According to the text features and the text mask, the initialized image features are T-ed using the stable diffusion model. 2 A denoising operation is performed to obtain a second final noise feature;

[0116] The second final noise feature is input into the decoder of the variational autoencoder of the stable diffusion model to obtain the poster base image.

[0117] In one embodiment, the generating module 114 is used to:

[0118] Extracting colors in the poster base image that occupy an area exceeding a preset ratio as candidate colors, wherein the candidate colors include white and black;

[0119] Determining a final text color of the placement area according to the placement area and the candidate colors;

[0120] The target poster image is generated according to the text content information, the placement area of ​​the text content, and the final text color of the placement area.

[0121] In one embodiment, the mask module 111 is specifically used for:

[0122] The background area in the poster template image is set to 0 and the text content is set to 1 to obtain a binary poster image;

[0123] Determine the minimum inscribed rectangle of the text content in the binary poster image, and set the value of the minimum inscribed rectangle to 1;

[0124] The text content is covered by the minimum inscribed rectangle to obtain the text mask image.

[0125] The invention provides a poster image generating device based on artificial intelligence. When a target poster image needs to be generated, a poster template image is selected, and text content information is input to generate a text mask image; and according to a user prompt input by a user for describing the target poster image in text, an intermediate image is generated by using a stable diffusion model; then the intermediate image is adjusted by using the text mask image and the stable diffusion model to generate a poster base image; finally, the text content information is added to the poster base image to obtain the target poster image.

[0126] In the embodiment of the present invention, to generate a target poster image, it is only necessary to select a poster template image, input text content information and user prompts, and then the target poster image can be generated. There is no need to introduce additional manpower or models for typesetting, which reduces resource consumption, shortens the time for generating the target poster image, and can generate poster images in batches. Furthermore, there is no need to introduce an additional control model to import poster layout information for the stable diffusion model, which reduces resource consumption.

[0127] For the specific definition of the poster image generation device based on artificial intelligence, please refer to the definition of the poster image generation method based on artificial intelligence above, which will not be repeated here. Each module in the above-mentioned poster image generation device based on artificial intelligence can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0128] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a poster image generation method based on artificial intelligence.

[0129] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a poster image generation method based on artificial intelligence

[0130] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0131] Generate a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0132] According to the user prompt, using the stable diffusion model, generating an intermediate image corresponding to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0133] According to the user prompt and the text mask image, the intermediate image is adjusted using the stable diffusion model to generate a poster base image of the target poster image;

[0134] The text content information is added to the poster base image to generate the target poster image. In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0135] Generate a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information;

[0136] According to the user prompt, using the stable diffusion model, generating an intermediate image corresponding to the user prompt, wherein the user prompt includes text description information of the target poster image;

[0137] According to the user prompt and the text mask image, the intermediate image is adjusted using the stable diffusion model to generate a poster base image of the target poster image;

[0138] The text content information is added to the poster base image to generate the target poster image.

[0139] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0141] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0142] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A poster image generation method based on artificial intelligence, characterized in that: include: Generate a text mask image according to the text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information; According to the user prompt, using the stable diffusion model, generating an intermediate image corresponding to the user prompt, wherein the user prompt includes text description information of the target poster image; According to the user prompt and the text mask image, the intermediate image is adjusted using the stable diffusion model to generate a poster base image of the target poster image; The text content information is added to the poster base image to generate the target poster image.

2. The method for generating poster images based on artificial intelligence as claimed in claim 1, characterized in that: The step of generating an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt comprises: Inputting the user prompt into the text encoder of the stable diffusion model to obtain text features; According to the text feature, perform T1 denoising operations on the initialized noise feature to obtain a first final noise feature, where T1 is a positive integer; The first final noise feature is input into a decoder of the variational autoencoder of the stable diffusion model to obtain the intermediate image.

3. The poster image generation method based on artificial intelligence as claimed in claim 2, characterized in that: The step of performing T1 denoising operations on the initialization noise feature according to the text feature to obtain the first final noise feature comprises: For the S-th denoising operation, the predicted noise corresponding to the S-1th time step and the text feature are input into the U-type network of the stable diffusion model to obtain the predicted noise corresponding to the S-th time step, wherein 1≤S≤T1, when S=1, the predicted noise corresponding to the S-1th time step is the initialization noise feature; The above steps are repeated T1 times, and the predicted noise output by the U-shaped network of the stable diffusion model is used as the first final noise feature.

4. The method for generating poster images based on artificial intelligence according to claim 1, characterized in that: The step of adjusting the intermediate image according to the user prompt and the text mask image and using the stable diffusion model to generate a poster base image of the target poster image comprises: Inputting the user prompt into the text encoder of the stable diffusion model to obtain text features; Inputting the preprocessed intermediate image into the encoder of the variational autoencoder of the stable diffusion model to obtain image features; Perform T2 noise addition operations on the image features to obtain initialized image features, where T2 is a positive integer; According to the text features and the text mask, using the stable diffusion model, performing T2 denoising operations on the initialized image features to obtain a second final noise feature; The second final noise feature is input into the decoder of the variational autoencoder of the stable diffusion model to obtain the poster base image.

5. The method for generating poster images based on artificial intelligence as claimed in claim 1, characterized in that ,, the poster template image includes a placement area for text content, and the steps of adding the text content information to the poster base image to generate the target poster image include: Extracting colors in the poster base image that occupy an area exceeding a preset ratio as candidate colors, wherein the candidate colors include white and black; Determining a final text color of the placement area according to the placement area and the candidate colors; The target poster image is generated according to the text content information, the placement area of ​​the text content, and the final text color of the placement area.

6. The method for generating poster images based on artificial intelligence as claimed in claim 5, characterized in that: The step of determining the final text color of the placement area according to the placement area and the candidate colors comprises: Calculating the contrast between the tile color of the placement area and each candidate color by a perceptual contrast algorithm; The alternative color with a contrast ratio greater than the preset contrast ratio is used as the target text color; The final text color is determined according to the target text color.

7. The method for generating poster images based on artificial intelligence according to any one of claims 1 to 6, characterized in that: The step of generating a text mask image according to the text content information of the selected poster template image and the target poster image comprises: The background area in the poster template image is set to 0 and the text content is set to 1 to obtain a binary poster image; Determine the minimum inscribed rectangle of the text content in the binary poster image, and set the value of the minimum inscribed rectangle to 1; The text content is covered by the minimum inscribed rectangle to obtain the text mask image.

8. A poster image generation device based on artificial intelligence, characterized in that: include: A mask module, used to generate a text mask image according to text content information of the selected poster template image and the target poster image, wherein the text content information includes main title information, subtitle information and text information; A synthesis module, configured to generate an intermediate image corresponding to the user prompt using the stable diffusion model according to the user prompt, wherein the user prompt includes text description information of the target poster image; A base map module, used to adjust the intermediate image according to the user prompt and the text mask image and to generate a poster base map of the target poster image by using the stable diffusion model; A generation module is used to add the text content information to the poster base image to generate the target poster image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the poster image generation method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the poster image generation method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

Images