Graph library generation method and device based on generative model, equipment and storage medium

Abstract

The invention relates to the technical field of image processing and the field of finance, and discloses a graphics library generation method and device based on a generative model, equipment and a storage medium, and the method comprises the steps: generating an initial advertisement picture corresponding to a marketing text abstract; initial advertisement picture semantic features and marketing text abstract semantic features are extracted through a semantic feature model, and semantic feature similarity is calculated; and determining the initial advertisement picture with the semantic feature similarity reaching a threshold as a target advertisement picture, and generating a target picture library according to the target advertisement picture. Through the mode, after the marketing text abstract is determined according to the marketing keyword, the marketing text abstract is sent into the picture generation model, and the initial advertisement picture related to the marketing text abstract is generated; and taking the picture of which the similarity between the semantic feature of the initial advertisement picture and the semantic feature of the marketing text abstract meets the requirement as a target advertisement picture, and establishing a target picture library according to the target advertisement picture, so that the diversity of pictures generated by an existing image generation model is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology and the financial field, and in particular to a method, device, equipment and storage medium for generating a gallery based on a generation model. Background Art

[0002] In the financial field, marketing requires a large amount of advertising, such as reaching potential customers by maintaining a circle of friends or sending advertising copy. In addition to pure text advertising copy, advertising copy formed by pictures that are semantically related to the promotional copy will undoubtedly arouse more user interest. Therefore, the selection of promotional pictures is the focus of each company's promotion.

[0003] At present, the types of images generated by traditional image generation solutions are unstable and difficult to match with the actual needs of users. The generated image types are very limited and difficult to meet the actual needs of users, and the diversity of image generation cannot be achieved. Therefore, how to improve the diversity of images generated by image generation models has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for generating a gallery based on a generative model to improve the diversity of images generated by the image generation model.

[0005] In a first aspect, the present application provides a method for generating a gallery based on a generative model, the method comprising:

[0006] Generate marketing text summaries based on marketing keywords;

[0007] Based on a preset image generation model, generating an initial advertising image corresponding to the marketing text summary;

[0008] Extracting the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features;

[0009] The initial advertisement pictures whose semantic feature similarity reaches a preset threshold are determined as target advertisement pictures, and a target picture library is generated according to the target advertisement pictures.

[0010] In a second aspect, the present application further provides a device for generating a gallery based on a generation model, the device comprising:

[0011] A marketing text summary generation module, used to generate marketing text summaries based on marketing keywords;

[0012] An initial advertisement picture generation module, used to generate an initial advertisement picture corresponding to the marketing text summary based on a preset picture generation model;

[0013] A semantic feature similarity module, used to extract the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculate the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features;

[0014] The picture library generation module is used to determine the initial advertising pictures whose semantic feature similarity reaches a preset threshold as target advertising pictures, and generate a target picture library based on the target advertising pictures.

[0015] In a third aspect, the present application also provides a device comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the above-mentioned generative model-based gallery generation method when executing the computer program.

[0016] In a fourth aspect, the present application further provides a storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the above-mentioned method for generating a gallery based on a generative model.

[0017] The present application discloses a method, device, equipment and storage medium for generating a picture library based on a generative model, wherein the picture library generating method based on a generative model includes generating a marketing text summary based on marketing keywords; generating an initial advertising picture corresponding to the marketing text summary based on a preset picture generation model; extracting the initial advertising picture semantic features of the initial advertising picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertising picture semantic features and the marketing text summary semantic features; determining the initial advertising picture whose semantic feature similarity reaches a preset threshold as a target advertising picture, and generating a target picture library based on the target advertising picture. Through the above-mentioned method, after determining the marketing text summary based on the marketing keywords, the present application sends the marketing text summary into the picture generation model to generate the initial advertising picture related to the marketing text summary, and then uses the pictures whose semantic features of the initial advertising picture and the semantic features of the marketing text summary meet the requirements as the target advertising pictures and establishes a target picture library based on them, which solves the problem of the high limitation of generating pictures through traditional technology when placing advertisements in the financial field, and improves the diversity of pictures generated by image generation models in the financial field. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic flow chart of a method for generating a gallery based on a generation model provided for the first embodiment of the present application;

[0020] Figure 2 A schematic flow chart of a method for generating a gallery based on a generation model provided for a second embodiment of the present application;

[0021] Figure 3 A schematic flow chart of a method for generating a gallery based on a generation model provided in the third embodiment of the present application;

[0022] Figure 4 A schematic flow chart of a method for generating a gallery based on a generation model provided in accordance with a fourth embodiment of the present application;

[0023] Figure 5 A schematic flow chart of a method for generating a gallery based on a generation model provided in accordance with a fifth embodiment of the present application;

[0024] Figure 6 A schematic flow chart of a method for generating a gallery based on a generation model provided for the sixth embodiment of the present application;

[0025] Figure 7 A schematic block diagram of a device for generating a picture library based on a generation model provided in an embodiment of the present application;

[0026] Figure 8 A schematic block diagram of the structure of a device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0030] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] The embodiments of the present application provide a method, device, equipment and storage medium for generating a gallery based on a generative model. The method for generating a gallery based on a generative model can be applied to a server. After determining the marketing text summary according to the marketing keywords, the marketing text summary is sent to the image generation model to generate an initial advertising image related to the marketing text summary. Then, the image whose semantic features of the initial advertising image meet the requirements of similarity with the semantic features of the marketing text summary is used as the target advertising image and a target gallery is established based on this. This solves the problem of the high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improves the diversity of images generated by image generation models in the financial field. The server can be an independent server or a server cluster.

[0032] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] See also Figure 1 , Figure 1 This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the first embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, thereby improving the diversity of images generated by the existing image generation model.

[0034] like Figure 1 As shown, the image gallery generation method based on the generation model specifically includes steps S10 to S40.

[0035] Step S10: Generate a marketing text summary based on the marketing keywords;

[0036] Step S20: generating an initial advertisement image corresponding to the marketing text summary based on a preset image generation model;

[0037] Step S30, extracting the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features;

[0038] Step S40: determining the initial advertisement pictures whose semantic feature similarity reaches a preset threshold as target advertisement pictures, and generating a target picture library according to the target advertisement pictures.

[0039] In a specific embodiment, when a traditional generation model generates an image through text, the text description often needs to be more detailed, but gallery users often want to use keywords as the theme to obtain pictures. Taking the banking financial management business as an example, marketing personnel hope to use appropriate copywriting with unique pictures in some specific seasons such as festivals and solar terms to send heartfelt greetings to potential customers in order to convert potential customers into real customers. However, searching for some pictures on the Internet will cause aesthetic fatigue in potential customers and fail to improve their impression on potential customers. However, through the gallery generation method involved in this embodiment, it is possible to effectively avoid the duplication of pictures with others, reflecting the diversity of pictures.

[0040] This embodiment obtains keywords input by the user, crawls related subject articles on the Internet through a crawler program, and then processes the related subject articles through a summary model to obtain a subject summary.

[0041] In this embodiment, the initial advertising image refers to at least one image, which is generated by a preset image generation model and is related to the marketing text summary. The image generation model can be a diffusion model, which is a model with a good image generation effect at present. The text is used as the input of the model and a picture of the corresponding content is generated. The summary model can be a PEGASUS (Pre-training with Extracted Gap-sentences for Abstractive Summarization, Pegasus) model or a UNIMO (unimodal distribution) model. In the pre-training stage, the important sentences of the input document are predicted by other sentences through the PEGASUS model or the UNIMO model to generate a summary.

[0042] In this embodiment, the user only needs to enter the desired marketing keywords to generate the corresponding gallery in a short time. The relevance and diversity of the pictures in this gallery are guaranteed, and there is no potential risk of infringing images, which is more friendly to commercial use including the financial industry.

[0043] The present embodiment discloses a method, device, equipment and storage medium for generating a picture gallery based on a generation model, wherein the method for generating a picture gallery based on a generation model includes generating a marketing text summary based on marketing keywords; generating an initial advertising picture corresponding to the marketing text summary based on a preset picture generation model and the marketing text summary; extracting the initial advertising picture semantic features of the initial advertising picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertising picture semantic features and the marketing text summary semantic features; determining the initial advertising picture whose semantic feature similarity reaches a preset threshold as a target advertising picture, and generating a target picture gallery based on the target advertising picture. Through the above-mentioned method, after determining the marketing text summary based on the marketing keywords, the present application sends the marketing text summary into the picture generation model to generate the initial advertising picture related to the marketing text summary, and then uses the pictures whose semantic features of the initial advertising picture and the semantic features of the marketing text summary meet the requirements as the target advertising pictures and establishes a target picture gallery based on them, which solves the problem of the high limitation of generating pictures by traditional technology when placing advertisements in the financial field, and improves the diversity of pictures generated by image generation models in the financial field.

[0044] See also Figure 2 , Figure 2 This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the second embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, which solves the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improves the diversity of images generated by image generation models in the financial field.

[0045] based on Figure 1 In the embodiment shown, in this embodiment, as Figure 2 As shown, step S30 specifically includes step S301 to step S302.

[0046] Step S301, determining an initial feature vector of the semantic feature of the initial advertisement image and a target feature vector of the semantic feature of the marketing text summary by using the preset semantic feature model;

[0047] Step S302: Calculate the Euclidean distance between the initial feature vector and the target feature vector to obtain the semantic feature similarity.

[0048] In a specific embodiment, the preset semantic feature model may be a language-image comparison pre-trained model, such as a textual inversion model, which may be used to extract the semantic features of the initial advertising image and calculate the similarity between the semantic information and the semantic features of the marketing keywords.

[0049] The present embodiment discloses a method, device, equipment and storage medium for generating a gallery based on a generative model. The method for generating a gallery based on a generative model includes determining the initial feature vector of the semantic features of the initial advertising image and the target feature vector of the semantic features of the marketing text summary through the preset semantic feature model; calculating the Euclidean distance between the initial feature vector and the target feature vector to obtain the similarity of the semantic features. In the above manner, the present application calculates the similarity between the initial feature vector and the target feature vector, and determines the similarity between the corresponding initial advertising image and the marketing keyword after obtaining the Euclidean distance, thereby solving the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improving the diversity of generating images through image generation models in the financial field.

[0050] See also Figure 3 , Figure 3 This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the third embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, which solves the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improves the diversity of images generated by image generation models in the financial field.

[0051] based on Figure 1 In the embodiment shown, in this embodiment, as Figure 3 As shown, step S40 also includes steps S41 to S42.

[0052] Step S41: determining whether the number of picture types corresponding to the target advertisement picture reaches a preset type threshold based on a preset diversity loss function;

[0053] Step S42: When the number of the image types does not reach the preset type threshold, the initial advertising image is updated by using the preset diversity loss function, the marketing text summary and the preset image generation model.

[0054] In a specific embodiment, based on a certain key indicator, a target distribution under the key indicator can be calculated through business experience or data. The target distribution is the ideal distribution. The diversity of the recommendation list can be measured by calculating the Hellinger distance between the recommendation list and the target distribution.

[0055] The semantic information of the initial advertising image is extracted through a preset semantic feature model, and then the semantic information of the marketing keywords is obtained. After the similarity is calculated based on the two, the similarity is compared with the similarity threshold, and the image collection with a similarity greater than the threshold is created as the target gallery.

[0056] The present embodiment discloses a method, device, equipment and storage medium for generating a picture gallery based on a generative model. The method for generating a picture gallery based on a generative model includes determining whether the number of picture types corresponding to the target advertising picture reaches a preset type threshold based on a preset diversity loss function; when the number of picture types does not reach the preset type threshold, updating the initial advertising picture through the preset diversity loss function, the marketing text summary and the preset picture generation model. In the above manner, the diversity loss function is used to determine whether the diversity of the currently generated picture meets the requirements for the diversity of generated pictures in the marketing process in the financial field. When the diversity does not meet the requirements, the marketing keywords are input into the picture generation model again, and the pictures are generated again until the diversity meets the preset requirements. This solves the problem of the high limitations of generating pictures through traditional technologies when placing advertisements in the financial field, and improves the diversity of pictures generated by image generation models in the financial field.

[0057] See also Figure 4 , Figure 4 This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the fourth embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, which solves the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improves the diversity of images generated by generative models in the financial field.

[0058] based on Figure 3 In the embodiment shown, in this embodiment, as Figure 4 As shown, step S41 includes steps S411 to S412.

[0059] Step S411, calculating the diversity distribution entropy value of the target advertising image based on the target relevance between the target advertising image and the marketing text summary, the Hellinger distance, and the preset diversity loss function;

[0060] Step S412: Based on the diversity distribution entropy value and a preset entropy value threshold, determine whether the number of image types reaches the preset type threshold.

[0061] Specifically, taking the financial management business as an example, when financial management marketers need to send out promotional pictures to a certain customer group, they can leave a deep impression on the customer group through appropriate copywriting and appropriate pictures. When the promotional pictures are generated through the gallery generation model, if the marketers think that the diversity is insufficient, they will generate pictures again through the generation model until the needs are met.

[0062] In a specific embodiment, the image generation model in this embodiment can be a language-image comparison pre-training model, and the images generated by this model may not be accurate enough. Therefore, this embodiment calculates the diversity distribution entropy value of the target advertising image after generating the image. When the diversity distribution entropy value is lower than the threshold, it is determined that the diversity of the generated image is insufficient, and then the marketing keywords are re-entered into the language-image comparison pre-training model to generate a batch of images again, and the cycle is repeated until the diversity of the images meets the threshold.

[0063] In probability and statistics, Hellinger Distance is used to measure the similarity between two probability distributions and is a type of f-divergence analysis.

[0064] The present embodiment discloses a method, device, equipment and storage medium for generating a gallery based on a generative model. The method for generating a gallery based on a generative model includes calculating the diversity distribution entropy of the target advertising image based on the target correlation between the target advertising image and the marketing text summary, the Hellinger distance and the preset diversity loss function; and determining whether the number of image types reaches the preset type threshold based on the diversity distribution entropy and the preset entropy threshold. In the above manner, the present application obtains the diversity distribution entropy by calculating the correlation and the Hellinger distance between the target advertising image and the target subject, and uses the diversity distribution entropy as a basis to compare with the entropy threshold to determine whether the diversity of the generated images meets the demand, thereby solving the problem of high limitations of generating images through traditional technology when placing advertisements in the financial field, thereby improving the diversity of images generated by image generation models in the financial field.

[0065] See also Figure 5 , Figure 5This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the fifth embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, which solves the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, thereby improving the diversity of images generated by image generation models in the financial field.

[0066] based on Figure 1 In the embodiment shown, in this embodiment, as Figure 5 As shown, after step S40, steps S43 to S44 are included.

[0067] Step S43, determining whether the number of the target advertising pictures in the target gallery reaches the target number;

[0068] Step S44: When the number of the target advertising pictures in the target gallery does not reach the target number, the initial advertising pictures are updated by using the marketing keywords and the preset picture generation model.

[0069] In a specific embodiment, take the advertising by marketing personnel in the financial field as an example. Different customer groups contain different attributes, such as age, gender, occupation, etc. Therefore, when placing advertisements or marketing promotions for different customer groups, we should try our best to cater to their preferences so that customers can leave a deep impression on marketing personnel or enterprises, which will help improve the success rate of cooperation. Then, according to different customer attributes, different copywriting and pictures are needed in publicity or online visits. The diversity and quantity of pictures are crucial at this time. In this embodiment, a sufficient number of pictures can be generated according to the keywords of marketing personnel through the gallery generation model, which is suitable for multiple publicity or multi-directional publicity, and helps to improve the success rate of cooperation.

[0070] The images generated by the preset language-image comparison pre-training model may lack diversity. A certain amount of images are obtained by inputting marketing keywords, but the types of these images will be affected by model training, resulting in low diversity. For example, when the marketing keyword input is "teacup", the generated images are all related to "teacup", but due to the constraints of the model itself, there may be fewer types of images. At this time, the preset diversity loss function can be used to obtain relevant keywords related to the marketing keywords, such as "mug", "glass", etc., and then these relevant keywords are input into the image generation model to obtain more images related to the marketing keywords and use them to establish the target gallery.

[0071] The present embodiment discloses a method, device, equipment and storage medium for generating a gallery based on a generative model. The method for generating a gallery based on a generative model includes determining whether the number of the target advertising images in the target gallery reaches the target number; when the number of the target advertising images in the target gallery does not reach the target number, the initial advertising images are updated through the marketing keywords and the preset image generation model. Through the above-mentioned method, after determining the marketing text summary according to the marketing keywords, the marketing text summary is sent to the generation model to generate the initial advertising images related to the marketing text summary, and then the initial semantic information of the initial advertising image is used as the classification basis to screen out the images that match the marketing text summary as the target advertising images. In order to ensure that the number of images in the gallery meets the demand, the number of target advertising images is used as the measurement standard. In the case of insufficient number of images, a batch of images are generated again through the generation model, which solves the problem of high limitations of generating images through traditional technology when placing advertisements in the financial field, and improves the diversity of images generated by the existing image generation model.

[0072] See also Figure 6 , Figure 6 This is a schematic flow chart of a method for generating a gallery based on a generative model provided in the sixth embodiment of the present application. The method for generating a gallery based on a generative model can be applied to a server, and is used to determine a marketing text summary based on marketing keywords, send the marketing text summary to a diffusion model, generate an initial gallery related to the marketing text summary, and then use the initial semantic information of the initial advertising image as a classification basis to screen out images that match the marketing text summary as target advertising images, and construct the target advertising images into a target gallery, which solves the problem of high limitations of generating images through traditional technologies when placing advertisements in the financial field, and improves the diversity of images generated by existing image generation models.

[0073] based on Figure 5 In the embodiment shown, in this embodiment, as Figure 6 As shown, step S44 includes step S441 to step S442.

[0074] Step S441: extracting the target semantic vector of the marketing keyword through the preset semantic feature model, and determining the relevant semantic vector associated with the target semantic vector;

[0075] Step S442: Based on the relevant semantic vector and the preset picture generation model, generate a relevant picture corresponding to the relevant semantic vector, and determine the relevant picture as the initial advertisement picture.

[0076] In a specific embodiment, for example, in insurance business, the contents of various insurance types are complex, and it is difficult for general users to quickly understand the differences between the various insurance types. At this time, the staff of the insurance company can provide keywords through the picture library generation model of this embodiment, and the picture library generation model automatically generates pictures according to the semantic vectors of the keywords, so that insurance users can quickly and accurately understand the specific contents of each insurance type, which is helpful for the insurance company and the insurance users to reach cooperation.

[0077] The diversity loss function can calculate the target distribution of an object under the key indicator through business experience or log data. The target distribution is the ideal distribution that takes diversity into consideration. The diversity of the recommendation list can be measured by calculating the Hellinger distance between the recommendation list and the target distribution. Diversity and relevance are taken into account in the loss function and solved by the gradient descent method. This method can make subtle adjustments to relevance and diversity.

[0078] The target semantic vector of the marketing keyword is obtained through the preset text feature extraction model, and the related semantic vector related to the target semantic vector is generated. The related semantic vector is then input into the preset diffusion model. Similarly, the related semantic vector is processed by the preset diffusion model to generate the corresponding related gallery. Since the semantic vector has been defined before, the related gallery is also associated with the target gallery, so the related gallery is added to the initial gallery to generate the target gallery.

[0079] The present embodiment discloses a method, device, equipment and storage medium for generating a gallery based on a generative model, wherein the method for generating a gallery based on a generative model comprises extracting the target semantic vector of the marketing keyword through the preset semantic feature model, and determining the relevant semantic vector associated with the target semantic vector; generating relevant pictures corresponding to the relevant semantic vector based on the relevant semantic vector and the preset picture generation model, and determining the relevant pictures as the initial advertising pictures. Through the above-mentioned method, after determining the marketing text summary according to the marketing keywords, the present application sends the marketing text summary into the picture generation model to generate the initial advertising picture related to the marketing text summary, and then uses the pictures whose semantic features of the initial advertising picture and the semantic features of the marketing text summary meet the requirements as the target advertising pictures and establishes a target gallery based on them, thereby improving the diversity of pictures generated by the existing image generation model.

[0080] Based on all the above embodiments, in this embodiment, step S10 includes:

[0081] Calculating keyword semantic information of the marketing keyword, and acquiring related topic articles based on the keyword semantic information;

[0082] The marketing text summary is generated by using a preset summary model and the related subject articles.

[0083] Specifically, the summary model can be a PEGASUS (Pre-training with Extracted Gap-sentences for Abstractive Summarization, Pegasus) model or a UNIMO (unimodal distribution) model. In the pre-training stage, the important sentences of the input document are predicted by other sentences through the PEGASUS model or the UNIMO model to generate a summary.

[0084] See also Figure 7 , Figure 7 The embodiment of the present application provides a schematic block diagram of a picture gallery generating device based on a generative model, wherein the picture gallery generating device based on a generative model is used to execute the aforementioned picture gallery generating method based on a generative model. The picture gallery generating device based on a generative model can be configured on a server.

[0085] like Figure 7 As shown, the image library generation device 400 based on the generation model includes:

[0086] A marketing text summary generation module 410, for generating a marketing text summary based on marketing keywords;

[0087] An initial advertisement picture generation module 420 is used to generate an initial advertisement picture corresponding to the marketing text summary based on a preset picture generation model;

[0088] A semantic feature similarity module 430 is used to extract the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculate the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features;

[0089] The picture library generation module 440 is used to determine the initial advertisement pictures whose semantic feature similarity reaches a preset threshold as target advertisement pictures, and generate a target picture library according to the target advertisement pictures.

[0090] Furthermore, the semantic feature similarity module 430 includes:

[0091] A feature vector extraction submodule, used to determine the initial feature vector of the semantic feature of the initial advertisement image and the target feature vector of the semantic feature of the marketing text summary through the preset semantic feature model;

[0092] The semantic feature similarity calculation submodule is used to calculate the Euclidean distance between the initial feature vector and the target feature vector to obtain the semantic feature similarity.

[0093] Furthermore, the image library generating device 400 based on the generation model further includes:

[0094] A picture type quantity determination module, used to determine whether the number of picture types corresponding to the target advertising picture reaches a preset type threshold based on a preset diversity loss function;

[0095] The first updating module for initial advertising pictures is used to update the initial advertising pictures by using the preset diversity loss function, the marketing text summary and the preset picture generation model when the number of picture types does not reach the preset type threshold.

[0096] Furthermore, the picture type quantity determination module includes:

[0097] A diversity distribution entropy value calculation unit, used to calculate the diversity distribution entropy value of the target advertising image based on the target relevance, the Hellinger distance and the preset diversity loss function between the target advertising image and the marketing text summary;

[0098] The picture type quantity determination unit is used to determine whether the number of picture types reaches the preset type threshold based on the diversity distribution entropy value and the preset entropy value threshold.

[0099] Furthermore, the image library generating device 400 based on the generation model further includes:

[0100] A picture quantity determination module, used to determine whether the number of the target advertising pictures in the target picture library reaches the target number;

[0101] The second updating module for initial advertising pictures is used to update the initial advertising pictures by using the marketing keywords and the preset picture generation model when the number of the target advertising pictures in the target gallery does not reach the target number.

[0102] Furthermore, the second updating module of the initial advertisement picture includes:

[0103] A semantic vector determination submodule, configured to extract a target semantic vector of the marketing keyword through the preset semantic feature model, and determine a related semantic vector associated with the target semantic vector;

[0104] The related picture generation submodule is used to generate a related picture corresponding to the related semantic vector based on the related semantic vector and the preset picture generation model, and determine the related picture as the initial advertisement picture.

[0105] Furthermore, the marketing text summary generation module 410 includes:

[0106] A related subject article acquisition submodule is used to calculate keyword semantic information of the marketing keyword and acquire related subject articles based on the keyword semantic information;

[0107] The marketing text summary generation submodule is used to generate the marketing text summary through a preset summary model and the related subject articles.

[0108] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0109] The above-mentioned device can be implemented in the form of a computer program. Figure 8 Runs on the devices shown.

[0110] See also Figure 8 , Figure 8 1 is a schematic block diagram of a device provided in an embodiment of the present application. The device may be a server.

[0111] See also Figure 8 The device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0112] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the image library generation methods based on the generation model.

[0113] The processor is used to provide computing and control capabilities to support the operation of the entire device.

[0114] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the image library generation methods based on the generation model.

[0115] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the device to which the scheme of the present application is applied. The specific device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0117] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0118] Generate marketing text summaries based on marketing keywords;

[0119] Based on a preset image generation model, generating an initial advertising image corresponding to the marketing text summary;

[0120] Extracting the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features;

[0121] The initial advertisement pictures whose semantic feature similarity reaches a preset threshold are determined as target advertisement pictures, and a target picture library is generated according to the target advertisement pictures.

[0122] In one embodiment, the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary are extracted by a preset semantic feature model, and the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features is calculated, so as to achieve:

[0123] Determine the initial feature vector of the semantic feature of the initial advertisement image and the target feature vector of the semantic feature of the marketing text summary by using the preset semantic feature model;

[0124] The Euclidean distance between the initial feature vector and the target feature vector is calculated to obtain the semantic feature similarity.

[0125] In one embodiment, after the initial advertisement picture whose semantic feature similarity reaches a preset threshold is determined as a target advertisement picture and a target gallery is generated according to the target advertisement picture, it is also used to implement:

[0126] Determining whether the number of image types corresponding to the target advertisement image reaches a preset type threshold based on a preset diversity loss function;

[0127] When the number of image types does not reach the preset type threshold, the initial advertisement image is updated by using the preset diversity loss function, the marketing text summary and the preset image generation model.

[0128] In one embodiment, determining whether the number of picture types corresponding to the target advertisement picture reaches a preset type threshold based on a preset diversity loss function is used to achieve:

[0129] Calculating the diversity distribution entropy value of the target advertising image based on the target relevance of the target advertising image and the marketing text summary, the Hellinger distance, and the preset diversity loss function;

[0130] Based on the diversity distribution entropy value and a preset entropy value threshold, it is determined whether the number of picture types reaches the preset type threshold.

[0131] In one embodiment, after the initial advertisement picture whose semantic feature similarity reaches a preset threshold is determined as a target advertisement picture and a target gallery is generated according to the target advertisement picture, it is also used to implement:

[0132] Determining whether the number of the target advertising images in the target gallery reaches the target number;

[0133] When the number of the target advertising pictures in the target gallery does not reach the target number, the initial advertising pictures are updated by using the marketing keywords and the preset picture generation model.

[0134] In one embodiment, when the number of the target advertising pictures in the target gallery does not reach the target number, the initial advertising pictures are updated by using the marketing keywords and the preset picture generation model to achieve:

[0135] Extracting a target semantic vector of the marketing keyword through the preset semantic feature model, and determining a related semantic vector associated with the target semantic vector;

[0136] Based on the relevant semantic vector and the preset picture generation model, a relevant picture corresponding to the relevant semantic vector is generated, and the relevant picture is determined as the initial advertisement picture.

[0137] In one embodiment, a marketing text summary is generated based on marketing keywords to achieve:

[0138] Calculating keyword semantic information of the marketing keyword, and acquiring related topic articles based on the keyword semantic information;

[0139] The marketing text summary is generated by using a preset summary model and the related subject articles.

[0140] A storage medium is also provided in an embodiment of the present application, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the generative model-based gallery generation methods provided in the embodiments of the present application.

[0141] The storage medium may be an internal storage unit of the device described in the foregoing embodiment, such as a hard disk or memory of the device. The storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the device.

[0142] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating a picture library based on a generative model, characterized in that: The method comprises: Generate marketing text summaries based on marketing keywords; Based on a preset image generation model, generating an initial advertising image corresponding to the marketing text summary; Extracting the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculating the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features; The initial advertisement pictures whose semantic feature similarity reaches a preset threshold are determined as target advertisement pictures, and a target picture library is generated according to the target advertisement pictures.

2. The image library generation method based on the generative model according to claim 1 is characterized in that: The extracting the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary by using a preset semantic feature model, and calculating the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features, includes: Determine the initial feature vector of the semantic feature of the initial advertisement image and the target feature vector of the semantic feature of the marketing text summary by using the preset semantic feature model; The Euclidean distance between the initial feature vector and the target feature vector is calculated to obtain the semantic feature similarity.

3. The image library generation method based on the generative model according to claim 1, characterized in that: After determining the initial advertisement picture whose semantic feature similarity reaches a preset threshold as a target advertisement picture and generating a target gallery according to the target advertisement picture, the method further includes: Determining whether the number of image types corresponding to the target advertisement image reaches a preset type threshold based on a preset diversity loss function; When the number of image types does not reach the preset type threshold, the initial advertisement image is updated by using the preset diversity loss function, the marketing text summary and the preset image generation model.

4. The method for generating a picture library based on a generative model according to claim 3, characterized in that: The determining, based on a preset diversity loss function, whether the number of picture types corresponding to the target advertisement picture reaches a preset type threshold includes: Calculating the diversity distribution entropy value of the target advertising image based on the target relevance of the target advertising image and the marketing text summary, the Hellinger distance, and the preset diversity loss function; Based on the diversity distribution entropy value and a preset entropy value threshold, it is determined whether the number of picture types reaches the preset type threshold.

5. The image library generation method based on the generative model according to claim 1, characterized in that: After determining the initial advertisement picture whose semantic feature similarity reaches a preset threshold as a target advertisement picture and generating a target gallery according to the target advertisement picture, the method further includes: Determining whether the number of the target advertising images in the target gallery reaches the target number; When the number of the target advertising pictures in the target gallery does not reach the target number, the initial advertising pictures are updated by using the marketing keywords and the preset picture generation model.

6. The method for generating a picture library based on a generative model according to claim 5, characterized in that: When the number of the target advertising pictures in the target gallery does not reach the target number, updating the initial advertising pictures by using the marketing keywords and the preset picture generation model includes: Extracting a target semantic vector of the marketing keyword through the preset semantic feature model, and determining a related semantic vector associated with the target semantic vector; Based on the relevant semantic vector and the preset picture generation model, a relevant picture corresponding to the relevant semantic vector is generated, and the relevant picture is determined as the initial advertisement picture.

7. The method for generating a picture library based on a generative model according to any one of claims 1 to 6, characterized in that: The generating of a marketing text summary based on marketing keywords includes: Calculating keyword semantic information of the marketing keyword, and acquiring related topic articles based on the keyword semantic information; The marketing text summary is generated by using a preset summary model and the related subject articles.

8. A device for generating a picture library based on a generative model, characterized in that: include: A marketing text summary generation module, used to generate marketing text summaries based on marketing keywords; An initial advertisement picture generation module, used to generate an initial advertisement picture corresponding to the marketing text summary based on a preset picture generation model; A semantic feature similarity module, used to extract the initial advertisement picture semantic features of the initial advertisement picture and the marketing text summary semantic features of the marketing text summary through a preset semantic feature model, and calculate the semantic feature similarity between the initial advertisement picture semantic features and the marketing text summary semantic features; The picture library generation module is used to determine the initial advertising pictures whose semantic feature similarity reaches a preset threshold as target advertising pictures, and generate a target picture library based on the target advertising pictures.

9. A device, characterized in that: The device comprises a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the image gallery generation method based on the generation model as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method for generating a gallery based on a generation model according to any one of claims 1 to 7.

Images