Advertisement image generation method and device, electronic equipment and storage medium

Abstract

The invention provides an advertisement image generation method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, image processing and the like, and the method comprises the steps: obtaining the input information of a to-be-pushed advertisement; according to the input information, a push prompt word of the to-be-pushed advertisement is determined, and the push prompt word comprises copywriting associated information of the to-be-pushed advertisement; the push prompt word is input into the first text graph model, the first text graph model generates the target advertisement image of the to-be-pushed advertisement according to the push prompt word, and the target advertisement image comprises the copywriting information of the to-be-pushed advertisement, so that the target advertisement image of the to-be-pushed advertisement is generated by the first text graph model according to the push prompt word; the generation efficiency of the advertisement image is improved, the requirements of users can be better met and understood, and the diversity and the visual effect of the advertisement image are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of large models and image processing, and in particular to a method, device, electronic device and storage medium for generating an advertising image. Background Art

[0002] In the related art, advertising images are often designed by designers or generated by text-based image models. However, current text-based image models are mostly trained on English data, resulting in poor performance of Chinese prompts, which cannot accurately express user needs and lack the ability to generate commercial marketing scenarios, and cannot fully meet the needs of advertisers. At the same time, advertising images usually contain advertising copy, that is, relevant text needs to be displayed in the image. Traditional methods often use templates to produce them, resulting in a lack of diversity in the generated advertising images and poor integration of the copy and the background image. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device, storage medium and computer program product for generating an advertisement image.

[0004] According to a first aspect of the present disclosure, a method for generating an advertising image is proposed, comprising: obtaining input information of an advertisement to be pushed; determining a push prompt word of the advertisement to be pushed based on the input information, wherein the push prompt word includes text-related information of the advertisement to be pushed; inputting the push prompt word into a first text-based graph model, and generating a target advertising image of the advertisement to be pushed based on the push prompt word by the first text-based graph model, wherein the target advertising image includes the text information of the advertisement to be pushed.

[0005] According to a second aspect of the present disclosure, a device for generating an advertising image is proposed, comprising: a first acquisition module, used to acquire input information of an advertisement to be pushed; a determination module, used to determine a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes text-related information of the advertisement to be pushed; and a second acquisition module, used to input the push prompt word into a first text-based graph model, and the first text-based graph model generates a target advertising image of the advertisement to be pushed according to the push prompt word, wherein the target advertising image includes the text information of the advertisement to be pushed.

[0006] According to a third aspect of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating an advertising image proposed in the first aspect above.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the method for generating an advertising image proposed in the first aspect.

[0008] According to a fifth aspect of the present disclosure, a computer program product is proposed, including a computer program, wherein when the computer program is executed by a processor, the method for generating an advertising image proposed in the first aspect is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 A schematic diagram of a flow chart of a method for generating an advertisement image according to an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of a flow chart of a method for generating an advertisement image according to an embodiment of the present disclosure;

[0013] Figure 3 A schematic diagram of a flow chart of a method for generating an advertisement image according to an embodiment of the present disclosure;

[0014] Figure 4 A schematic diagram of training the first text-generated graph model and the second text-generated graph model according to an embodiment of the present disclosure;

[0015] Figure 5 It is a structural schematic diagram of a device for generating an advertisement image according to an embodiment of the present disclosure;

[0016] Figure 6 A schematic block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0018] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0019] A large model refers to a machine learning model with a large parameter scale and complexity. It requires a large amount of computing resources and storage space to train and store, and often requires distributed computing and special hardware acceleration technology. Large models have stronger generalization and expression capabilities.

[0020] Image processing is the technology of using computers to analyze images to achieve the desired results. Image processing generally refers to digital image processing. Digital images refer to a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of the array are called pixels, and their values ​​are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.

[0021] Figure 1 FIG. 1 is a flow chart of a method for generating an advertisement image according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0022] S101: Obtain input information of the advertisement to be pushed.

[0023] It should be noted that the execution subject of the method for generating an advertising image in the embodiment of the present disclosure may be a hardware device capable of generating a digital advertising image and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other intelligent devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, etc.

[0024] The input information of the advertisement to be pushed can be used to represent the demand information of the advertiser.

[0025] Among them, advertisers refer to legal persons, other economic organizations or individuals who design, produce and publish advertisements by themselves or entrust others to do so in order to promote goods or provide services. Advertisers are the initiators of advertising activities, merchants who sell or promote their own products and services online, and providers of affiliate marketing advertisements. Any merchant who promotes and sells its products or services can serve as an advertiser.

[0026] Optionally, the advertiser may obtain candidate information related to the advertisement to be pushed, and select target information from the candidate information related to the advertisement to be pushed as input information of the advertisement to be pushed.

[0027] For example, candidate information related to the advertisement to be pushed includes, but is not limited to, information such as the advertisement theme, target audience, and advertisement features.

[0028] It should be noted that the present disclosure does not limit the form of the input information. For example, the input information may be a word, a sentence, a text paragraph, etc.

[0029] S102: Determine a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes copy-related information of the advertisement to be pushed.

[0030] The push prompt words include but are not limited to instruction information and data information. The instruction information is used to indicate the task target of the first text-generated image model, and the data information includes but is not limited to input information, etc.

[0031] It should be noted that by determining the push prompt words of the advertisement to be pushed according to the input information, it is possible to better understand and generate an advertisement image that meets the needs of the advertiser, and to achieve personalized customization of the advertisement image.

[0032] In the embodiment of the present disclosure, after the input information is acquired, key information of the advertisement to be pushed may be extracted from the input information, and a push prompt word of the advertisement to be pushed may be constructed based on the extracted key information.

[0033] S103, inputting the push prompt word into the first text-generated image model, and the first text-generated image model generates a target advertisement image of the advertisement to be pushed according to the push prompt word, wherein the target advertisement image includes text information of the advertisement to be pushed.

[0034] Among them, the Text-to-Image Generation model can convert natural language text into image space, link visual features with language information, and realize the mapping between natural language text and images.

[0035] In an embodiment of the present disclosure, a first sample advertising image including copy information can be obtained, first sample description information of the first sample advertising image can be obtained, the first sample description information including the copy information on the first sample advertising image, the copy information on the first sample advertising image is determined as an auxiliary control condition, the first sample advertising image and the first sample description information are used as a first sample image-text pair, and based on the first sample image-text pair, the second text-generated image model is fine-tuned and trained to obtain a first text-generated image model.

[0036] It should be noted that the push prompt words are parsed to obtain the push demand information in the push prompt words, and the first text-generated image model generates the target advertisement image of the advertisement to be pushed based on the push demand information.

[0037] In an embodiment of the present disclosure, after the push prompt words are obtained, the push prompt words can be input into the first text-generated image model, and the first text-generated image model generates an advertising base map according to the push demand information in the push prompt words, and predicts the layout information of the copy information in the advertising base map, and adds the copy information to the advertising base map based on the layout information to output the target advertising image.

[0038] It should be noted that the layout information includes but is not limited to the position layout information, the level layout information, the visual layout information, etc. of the copy information.

[0039] Optionally, after adding the copy information to the advertising base map based on the layout information, in order to improve the quality of the target advertising image, the advertising image parameters can be optimized, for example: optimizing the brightness parameters and contrast parameters of the advertising image. After the advertising image parameters are optimized, the target advertising image is output.

[0040] The method for generating an advertising image proposed in the present disclosure obtains input information of an advertisement to be pushed, determines a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes text-related information of the advertisement to be pushed, inputs the push prompt word into a first text-generated image model, and generates a target advertising image of the advertisement to be pushed according to the push prompt word by the first text-generated image model, wherein the target advertising image includes text information of the advertisement to be pushed. Thus, the method for generating an advertising image according to the push prompt word by the first text-generated image model is improved, and the needs of users can be better met and understood. The text information and the advertising background image can be better integrated, thereby improving the diversity and visual effects of the advertising images.

[0041] Figure 2 It is a flowchart of a method for generating an advertisement image according to the second embodiment of the present disclosure.

[0042] like Figure 2 As shown, in Figure 2 Based on the illustrated embodiment, the method of obtaining the first text graph model in the embodiment of the present disclosure may specifically include the following steps:

[0043] S201: Acquire a first sample advertisement image including text information.

[0044] In an embodiment of the present disclosure, an advertising image set may be collected, optical character recognition (OCR) may be performed on sample advertising images in the advertising image set, and the advertising image set may be grouped based on the OCR recognition results to obtain a first image set, wherein the first image set includes candidate first sample advertising images.

[0045] For example, after the OCR recognition result of the sample advertisement image is obtained, the advertisement images including text in the sample advertisement image are divided into the first image set based on the OCR recognition result.

[0046] In the embodiments of the present disclosure, in order to ensure the quality of the first sample advertising images (training data), image quality information of candidate sample advertising images in the first image set can be obtained, and sample advertising images for model training can be screened from the candidate sample advertising images in the first image set based on the image quality information.

[0047] Optionally, quality assessment may be performed on the candidate sample advertisement images in the first image set to obtain image quality information of the candidate sample advertisement images in the first image set.

[0048] It should be noted that the present disclosure does not limit the type of image quality information, which can be selected according to actual conditions.

[0049] Optionally, the image quality information may be image quality information; optionally, the image quality information may be image resolution information.

[0050] S202: Acquire first sample description information of a first sample advertisement image, where the first sample description information includes text information on the first sample advertisement image.

[0051] In the disclosed embodiment, language understanding may be performed on the first sample advertisement image based on a pre-trained large language model to obtain first sample description information corresponding to the first sample advertisement image.

[0052] For example, a multimodal large language model may be pre-trained based on the training data, and the first sample description information caption1 corresponding to the first sample advertisement image may be obtained through the pre-trained large language model according to the first sample advertisement image.

[0053] S203: Determine the text information on the first sample advertisement image as an auxiliary control condition.

[0054] It should be noted that, in the training process of the first text-generated image model, by taking the text information on the first sample advertising image as an auxiliary control condition, it can be understood as introducing a control mechanism control-net. Taking the text information as an auxiliary control condition can make the advertising image generated by the first text-generated image model include the advertising text information.

[0055] Optionally, during the training process of the first text-generated image model, by introducing the attention mechanism cross-attention, the correlation between different modalities (text and image) can be captured, thereby generating an advertising image that matches the push prompt word.

[0056] S204, taking the first sample advertisement image and the first sample description information as a first sample image-text pair, and based on the first sample image-text pair, fine-tuning the second text-generated image model to obtain a first text-generated image model.

[0057] In an embodiment of the present disclosure, after obtaining the first sample advertising image and the first sample description information, the first sample advertising image and the first sample description information can be combined to form a first sample image-text, and the first sample image-text pair is used as training data. Based on the first sample image-text, the second text-generated image model is fine-tuned and trained to obtain the first text-generated image model.

[0058] It should be noted that in the training process of the first text-image model, auxiliary control conditions are injected into the second text-image model by introducing the control mechanism control-net and the attention mechanism cross-attention. Based on the first sample image-text pair, the second text-image model is fine-tuned and trained, so that the first text-image model has the ability to generate written text.

[0059] The method for generating an advertising image proposed in the present disclosure obtains a first sample advertising image including copy information, obtains first sample description information of the first sample advertising image, the first sample description information includes copy information on the first sample advertising image, determines the copy information on the first sample advertising image as an auxiliary control condition, takes the first sample advertising image and the first sample description information as a first sample image-text pair, and fine-tunes the second text-generated image model based on the first sample image-text pair to obtain a first text-generated image model. Therefore, the present disclosure fine-tunes the second text-generated image model based on the first sample image-text pair to obtain the first text-generated image model, so that the first text-generated image model can generate an advertising image including advertising copy information according to a prompt word, thereby improving the generation efficiency of advertising images, better meeting and understanding user needs, and improving the diversity and visual effects of advertising images.

[0060] Figure 3 It is a flowchart of a method for generating an advertisement image according to the third embodiment of the present disclosure.

[0061] like Figure 3 As shown, in Figure 3 Based on the illustrated embodiment, the method of obtaining the second text graph model in the embodiment of the present disclosure may specifically include the following steps:

[0062] S301: Acquire a second sample advertisement image.

[0063] In an embodiment of the present disclosure, an advertising image set may be collected, OCR recognition may be performed on sample advertising images in the advertising image set, and the advertising image set may be grouped based on the OCR recognition results to obtain a second image set, wherein the second image set includes candidate second sample advertising images.

[0064] For example, after the OCR recognition result of the sample advertisement image is obtained, the advertisement images that do not include text in the sample advertisement image are divided into the second image set based on the OCR recognition result.

[0065] In the disclosed embodiment, in order to ensure the quality of the second sample advertising images (training data), image quality information of candidate sample advertising images in the second image set can be obtained, and sample advertising images for model training can be screened from the candidate sample advertising images in the second image set based on the image quality information.

[0066] Optionally, quality assessment may be performed on the candidate sample advertisement images in the second image set to obtain image quality information of the candidate sample advertisement images in the second image set.

[0067] It should be noted that the present disclosure does not limit the type of image quality information, which can be selected according to actual conditions.

[0068] Optionally, the image quality information may be image quality information; optionally, the image quality information may be image resolution information.

[0069] S302: Obtain second sample description information of the second sample advertisement image.

[0070] In the disclosed embodiment, language understanding may be performed on the second sample advertisement image based on a pre-trained large language model to obtain second sample description information corresponding to the second sample advertisement image.

[0071] For example, a multimodal large language model may be pre-trained based on the training data, and the second sample description information caption2 corresponding to the second sample advertisement image may be obtained through the pre-trained large language model according to the second sample advertisement image.

[0072] S303: Using the second sample advertisement image and the second sample description information as the second sample image-text pair, training the base text-image model to obtain a second text-image model.

[0073] In the embodiment of the present disclosure, after obtaining the second sample advertising image and the second sample description information, a second sample image-text pair can be formed based on the second sample advertising image and the second sample description information, and the second sample image-text pair can be used as training data to train the base text image model to obtain a second text image model.

[0074] Among them, the base text map model can be a diffusion model based on Transformer (DiffusionTransformer, referred to as DiT).

[0075] Optionally, the base cultural graph model can be expanded, that is, the number of parameters of the base cultural graph model can be increased, and the base cultural graph model can be trained based on the optimization strategy of distributed training to improve the training efficiency of the base cultural graph model, enhance the second cultural graph model's ability to accommodate larger-scale data, and enhance the second cultural graph model's learning and expression capabilities.

[0076] In an embodiment of the present disclosure, a second sample image-text pair may be sent to a distributed training node, which may perform joint training on some model parameters of the base cultural graph model based on the second sample image-text pair until the target model parameters are obtained after the training is completed, and obtain the second cultural graph model based on the target model parameters trained by the distributed training node.

[0077] In an embodiment of the present disclosure, a sample text in a first language is obtained, and the sample text in the first language is translated to obtain a sample text in a second language. Based on the sample text in the first language, the sample text in the second language and the target image, a text encoder of a base text model is trained to obtain a text encoder that can understand the second language.

[0078] It should be noted that the present disclosure does not limit the settings of the first language sample text and the second language sample text. Optionally, the first language sample text is an English sample text, and the second language sample text is a Chinese sample text.

[0079] In an embodiment of the present disclosure, a noise image predicted by a base text graph model in a forward diffusion process is obtained, the noise image, a sample text in a first language, and a sample text in a second language are aligned, and input into an image generator in the base text graph model for back diffusion processing to obtain a predicted image, and the loss of the predicted image and the target image is obtained to adjust the base text graph model until the training is completed to obtain a text encoder that can understand the second language.

[0080] Optionally, when the loss between the predicted image and the target image is less than a preset loss threshold, the training ends and a text encoder that can understand the second language is obtained.

[0081] The method for generating an advertising image proposed in the present disclosure obtains a second sample advertising image, obtains second sample description information of the second sample advertising image, uses the second sample advertising image and the second sample description information as a second sample image-text pair, trains a base text-image model, and obtains a second text-image model. Thus, the present disclosure trains the base text-image model through the second sample image-text pair to obtain the second text-image model, thereby realizing the ability transfer from generating a target advertising image from prompt words of a first language sample text to generating a target advertising image from prompt words of a second language sample text, enhancing the text control ability and image generation quality of the second text-image model, and laying a foundation for more accurate subsequent acquisition of the first text-image model.

[0082] The training process of the first text-generated graph model and the second text-generated graph model proposed in the embodiment of the present disclosure is explained below.

[0083] For example, Figure 4 As shown, the training process of the first text-generated graph model and the second text-generated graph model may include the following stages:

[0084] For stage (a), image captioner training: use a high-quality image as the second sample advertising image, input the second sample advertising image into an image encoder, obtain the encoding result of the second sample advertising image, input the encoding result of the second sample advertising image into the Q-former module, input the prompt word prompt and the output of the Q-former into the large language model (LLM) to perform language understanding on the second sample advertising image, and obtain the second sample descriptive information (descriptive Chinese captions) corresponding to the second sample advertising image.

[0085] For stage (b), the encoder alignment stage (encoder alignment), obtain the second language sample text (Chinese translation sample text Chinese translations and the first language sample text (English sample text English texts), input the second language sample text and the first language sample text into a text encoder (text encoder), wherein the text encoder includes a 128 text encoder and a text encoder (English text encoder) that can understand the first language (English), and determine the mean square error (MSE) loss loss based on the output of the 128 text encoder and the output of the English text encoder. The second sample advertising image and the second sample description information are used as the second sample image-text pair, and the base text image model is trained to obtain the second text image model, that is, the second sample description information (Chinese text) is input into the 128 text encoder, and the second sample advertising image is input into the image encoder (image encoder), and the contrast loss is determined based on the output of the 128 text encoder and the output of the image encoder. The base text image model is trained based on the loss to obtain the second text image model. For the specific process of the training, please refer to the above embodiment, which will not be repeated here.

[0086] For stage (c), the progressive image generator training stage (progressive image generator training), obtain the noise image predicted by the base cultural image model in the forward diffusion process, take the first sample advertising image and the first sample description information (descriptive Chinese captions) as the first sample image-text pair, obtain the noise image predicted by the base cultural image model in the forward diffusion process, input the first sample description information into the 128 text encoder, fix some model parameters (Frozen), obtain the reconstruction loss (reconstruction loss) of the sample image, input the first sample description information (descriptive Chinese captions) into the 128 text encoder, optimize some model parameters (Tuned), fine-tune the second cultural image model based on the reconstruction loss, and obtain the first cultural image model. For the specific process of training, please refer to the above embodiment, which will not be repeated here.

[0087] In summary, the first text-generated image model generates an advertising base map according to the push demand information in the push prompt words. The most advanced diffusion model is used, and the distillation method is used to enable it to have the ability to generate Chinese. After fine-tuning on high-quality images, the base map can be directly generated according to the advertiser's text prompt word prompt, and the advertising copy can be automatically added. It can more accurately understand and generate prompts that conform to the Chinese context, thereby improving the user's ability to express needs, and can better understand and generate content that meets the needs of advertisers, improve the relevance and attractiveness of advertising content, reduce the workload of manual template production, improve production efficiency, achieve a better integration of copy and base map, enhance the overall visual effect and attractiveness of advertising images, and at the same time break through the limitations of traditional templates and provide more diverse design options.

[0088] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0089] According to an embodiment of the present disclosure, the present disclosure also provides a device for generating an advertisement image, which is used to implement the above-mentioned method for generating an advertisement image.

[0090] Figure 5 The invention is a block diagram of a device for generating an advertisement image according to an embodiment of the present invention.

[0091] like Figure 5 As shown, the advertisement image generating device 500 includes: a first obtaining module 501 , a determining module 502 and a second obtaining module 503 .

[0092] The first acquisition module 501 is used to acquire input information of the advertisement to be pushed;

[0093] A determination module 502 is used to determine a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes copy-related information of the advertisement to be pushed;

[0094] The second acquisition module 503 is used to input the push prompt word into the first text image model, and the first text image model generates a target advertisement image of the advertisement to be pushed according to the push prompt word, and the target advertisement image includes the text information of the advertisement to be pushed.

[0095] In one embodiment of the present disclosure, the second acquisition module 503 is further used to: generate an advertising base map according to the push demand information in the push prompt word by the first text-graph model, predict layout information of the text information in the advertising base map, and add the text information to the advertising base map based on the layout information to output the target advertising image.

[0096] In one embodiment of the present disclosure, the device 500 is also used to: obtain a first sample advertising image including copy information; obtain first sample description information of the first sample advertising image, wherein the first sample description information includes copy information on the first sample advertising image; determine the copy information on the first sample advertising image as an auxiliary control condition; use the first sample advertising image and the first sample description information as a first sample image-text pair, and based on the first sample image-text pair, fine-tune the second text-generated image model to obtain the first text-generated image model.

[0097] In one embodiment of the present disclosure, the process of acquiring the second cultural image model includes: acquiring a second sample advertising image; acquiring second sample description information of the second sample advertising image; and training the base cultural image model with the second sample advertising image and the second sample description information as a second sample image-text pair to obtain the second cultural image model.

[0098] In one embodiment of the present disclosure, the device 500 is also used to: send the second sample image-text pair to a distributed training node; the distributed training node jointly trains some model parameters of the base cultural graph model according to the second sample image-text pair until the training is completed to obtain the target model parameters; and obtain the second cultural graph model based on the target model parameters trained by the distributed training node.

[0099] In one embodiment of the present disclosure, the device 500 is further used to: obtain a first language sample text, and translate the first language sample text to obtain a second voice sample text; based on the first language sample text, the second language sample text and the target image, train the base text model text encoder to obtain a text encoder that can understand the second language.

[0100] In one embodiment of the present disclosure, the device 500 is further used to: obtain a noise image predicted by the base text graph model in a forward diffusion process; align the noise image, the first language sample text, and the second language sample text, and input them into an image generator in the base text graph model for back diffusion processing to obtain a predicted image; obtain the loss of the predicted image and the target image to adjust the base text graph model until the training is completed to obtain the text encoder that can understand the second language.

[0101] In one embodiment of the present disclosure, the device 500 is also used to: for any advertising image among the first sample advertising image and the second sample advertising image, perform language understanding on the any advertising image based on a pre-trained large language model to obtain sample description information corresponding to the any advertising image.

[0102] In one embodiment of the present disclosure, the device 500 is also used to: collect an advertising image set; perform optical character OCR recognition on sample advertising images in the advertising image set; and group them based on the OCR recognition results to obtain a first image set and a second image set, wherein the first image set includes candidate first sample advertising images, and the second image set includes candidate second sample advertising images.

[0103] In one embodiment of the present disclosure, the device 500 is further used to: obtain image quality information of candidate sample advertising images in any image set in the first image set and the second image set; and screen sample advertising images for model training from the candidate sample advertising images in any image set based on the image quality information.

[0104] The device for generating an advertisement image proposed in the present disclosure obtains input information of an advertisement to be pushed, determines a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes text-related information of the advertisement to be pushed, inputs the push prompt word into a first text-generated graph model, and generates a target advertisement image of the advertisement to be pushed according to the push prompt word by the first text-generated graph model, wherein the target advertisement image includes text information of the advertisement to be pushed. Thus, the device for generating an advertisement image of the advertisement to be pushed according to the push prompt word by the first text-generated graph model in the present disclosure improves the generation efficiency of advertisement images, can better meet and understand the needs of users, can better integrate text information and advertisement background images, and improves the diversity and visual effects of advertisement images.

[0105] According to an embodiment of the present disclosure, the present disclosure also proposes an electronic device, a readable storage medium, and a computer program product.

[0106] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0107] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0108] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0109] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the method for generating an advertising image. For example, in some embodiments, the method for generating an advertising image may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for generating an advertising image described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method for generating an advertising image in any other appropriate manner (e.g., by means of firmware).

[0110] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be presented to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0112] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To propose interactions with a user account, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user account can propose input to the computer. Other types of devices may also be used to propose interactions with the user account; for example, feedback proposed to the user account may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user account may be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user account computer with a graphical user account interface or a web browser through which a user account can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0115] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0116] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the method for generating an advertising image described in the above embodiment of the present disclosure are implemented.

[0117] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0118] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating an advertising image, wherein: The method comprises: Get input information of the advertisement to be pushed; Determine, according to the input information, a push prompt word of the advertisement to be pushed, wherein the push prompt word includes copy-related information of the advertisement to be pushed; The push prompt word is input into a first text-image model, and the first text-image model generates a target advertisement image of the advertisement to be pushed according to the push prompt word, wherein the target advertisement image includes text information of the advertisement to be pushed.

2. The method according to claim 1, wherein: The step of generating the target advertisement image of the advertisement to be pushed according to the push prompt word by the first text graph model includes: The first text-image model generates an advertising base map according to the push demand information in the push prompt word, predicts the layout information of the text information on the advertising base map, and adds the text information to the advertising base map based on the layout information to output the target advertising image.

3. The method according to claim 1 or 2, wherein: The method further comprises: Acquire a first sample advertisement image including copy information; Acquire first sample description information of the first sample advertising image, where the first sample description information includes text information on the first sample advertising image; Determining the text information on the first sample advertisement image as an auxiliary control condition; The first sample advertisement image and the first sample description information are used as a first sample image-text pair, and based on the first sample image-text pair, a second text-generated image model is fine-tuned and trained to obtain the first text-generated image model.

4. The method according to claim 3, wherein: The process of obtaining the second text graph model includes: Acquire a second sample advertisement image; Acquire second sample description information of the second sample advertisement image; The second sample advertising image and the second sample description information are used as the second sample image-text pair to train the base text-image model to obtain the second text-image model.

5. The method according to claim 4, wherein: The method further comprises: Sending the second sample image-text pair to the distributed training node; The distributed training node jointly trains part of the model parameters of the base text graph model according to the second sample text-graph pair until the training is completed to obtain the target model parameters; The second cultural graph model is obtained based on the target model parameters trained by the distributed training nodes.

6. The method according to claim 4, wherein: The method further comprises: Acquire a first language sample text, and translate the first language sample text to obtain a second voice sample text; Based on the sample text in the first language, the sample text in the second language and the target image, the base text model text encoder is trained to obtain a text encoder that can understand the second language.

7. The method according to claim 6, wherein: The step of training the text encoder in the base text graph model based on the first language sample text and the second language sample text to obtain a text encoder that can understand the second language includes: Obtaining a noise image predicted by the base viz map model during the forward diffusion process; Aligning the noise image, the sample text in the first language, and the sample text in the second language, and inputting them into the image generator in the base text graph model for back diffusion processing to obtain a predicted image; The losses of the predicted image and the target image are obtained to adjust the base text graph model until the training is completed to obtain the text encoder that can understand the second language.

8. The method according to claim 3, wherein: The method further comprises: For any one of the first sample advertising image and the second sample advertising image, language understanding is performed on the any one of the advertising images based on a pre-trained large language model to obtain sample description information corresponding to the any one of the advertising images.

9. The method according to claim 3, wherein: The method further comprises: Collecting a collection of advertising images; Performing optical character recognition (OCR) on sample advertisement images in the advertisement image set; The OCR recognition results are grouped to obtain a first image set and a second image set, wherein the first image set includes candidate first sample advertisement images, and the second image set includes candidate second sample advertisement images.

10. The method according to claim 9, wherein: The method further comprises: For any image set in the first image set or the second image set, obtaining image quality information of a candidate sample advertisement image in the any image set; According to the image quality information, sample advertisement images for model training are screened from candidate sample advertisement images in any one of the image sets.

11. A device for generating an advertising image, wherein: The device comprises: A first acquisition module, used to acquire input information of the advertisement to be pushed; A determination module, configured to determine a push prompt word of the advertisement to be pushed according to the input information, wherein the push prompt word includes copy-related information of the advertisement to be pushed; The second acquisition module is used to input the push prompt word into the first text image model, and the first text image model generates a target advertisement image of the advertisement to be pushed according to the push prompt word, wherein the target advertisement image includes the text information of the advertisement to be pushed.

12. An electronic device, characterized in that: including a processor and a memory; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images