An image generation method, apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide an image generation method and device, electronic equipment and storage medium, and relate to the technical field of image processing. The method comprises: identifying a use intention of an original image and extracting a target keyword representing the use intention based on the semantics of the text in the original image; extracting visual features of the original image; determining fusion parameters based on the visual features; wherein the fusion parameters are used to adapt the visual style of the original image and make the keyword highlighted in the original image; and fusing the target keyword in the original image based on the fusion parameters. In this way, the image generation efficiency and accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image generation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Various service platforms, such as those for video and product sales, can provide online resources such as videos or product detail pages. When users need to browse these online resources, the service platform can display images of the online resources to attract users to click on them and browse.

[0003] In existing technologies, technicians can analyze the information displayed in the original image of a network resource to identify corresponding keywords. For example, they can use the text contained in the original image as keywords, or summarize the keywords after manually understanding the text in the original image. Then, the corresponding keywords are added to the original image of the network resource to generate a target image for resource recommendation.

[0004] However, according to existing technology, the above processing needs to be performed manually for each network resource in turn, which is inefficient. Furthermore, the determination of keywords is easily affected by human subjectivity, ultimately resulting in low efficiency and accuracy in generating target images. Summary of the Invention

[0005] The purpose of this invention is to provide an image generation method, apparatus, electronic device, and storage medium to improve image generation efficiency and accuracy. The specific technical solution is as follows:

[0006] In a first aspect of the present invention, an image generation method is provided, the method comprising:

[0007] Based on the semantics of the text in the original image, the intended use of the original image is identified, and target keywords representing the intended use are extracted;

[0008] Extract the visual features of the original image;

[0009] Based on the visual features, fusion parameters are determined; wherein, the fusion parameters are used to adapt to the visual style of the original image and to highlight the keywords in the original image;

[0010] Based on the fusion parameters, the target keywords are fused into the original image.

[0011] Optionally, the step of identifying the usage intent of the original image based on the semantics of the text in the original image and extracting target keywords characterizing the usage intent includes:

[0012] Extract the text from the original image to obtain the original text;

[0013] Semantic analysis is performed on the original text to identify the intended use of the original image and extract candidate keywords that represent the intended use; and based on the predicted click probability of the image displaying each candidate keyword, the predicted click probability corresponding to that candidate keyword is obtained.

[0014] From all the candidate keywords, at least one candidate keyword with the highest predicted click probability is selected as the target keyword representing the intended use.

[0015] Optionally, obtaining the predicted click probability corresponding to a candidate keyword based on the predicted click probability of the image displaying each candidate keyword includes:

[0016] If there is a preset keyword whose similarity to each candidate keyword is greater than a preset threshold, then the preset click probability corresponding to the preset keyword is used as the predicted click probability corresponding to that candidate keyword; wherein, the preset click probability corresponding to each preset keyword represents the click frequency of the image displaying that preset keyword;

[0017] If the preset keyword does not exist, the candidate click probability corresponding to the candidate keyword output by the prediction model is obtained as the predicted click probability of the candidate keyword; wherein, the prediction model is trained based on sample text, sample keywords in the sample text, and the actual click probability corresponding to each sample keyword.

[0018] Optionally, fusing the target keywords into the original image based on the fusion parameters includes:

[0019] A watermark base is generated based on the first parameter in the fusion parameters that describes the overall style of the original image.

[0020] The target keywords are adjusted according to the second parameter in the fusion parameters that describes the visual morphology of the keywords in the image;

[0021] The adjusted target keywords are rendered on the watermark base to obtain a fused watermark carrying the target keywords;

[0022] The fusion watermark is fused into the original image based on the third parameter in the fusion parameters that describes the location of the keywords.

[0023] Optionally, the first parameter includes: the main color tone of the original image, the overall brightness information of the original image, and the style type of the original image;

[0024] The step of generating a watermark base based on the first parameter describing the overall style of the original image in the fusion parameters includes:

[0025] Based on the primary color tone and overall brightness information contained in the first parameter of the fusion parameters, the color and transparency of the watermark base are determined.

[0026] Generate the watermark base to be processed according to the determined color and transparency;

[0027] Determine the texture corresponding to the style type in the fusion parameters;

[0028] A defined texture is added to the watermark base to obtain the desired watermark base.

[0029] Optionally, the third parameter is obtained in the following manner:

[0030] Identify a preset protected area in the original image;

[0031] In the original image, outside the preset protected area, the location of the keyword is determined, and a third parameter describing the location of the keyword is obtained.

[0032] In a second aspect of the invention, an image generation apparatus is also provided, the apparatus comprising:

[0033] The first extraction module is used to identify the usage intent of the original image based on the semantics of the text in the original image, and extract target keywords that characterize the usage intent;

[0034] The second extraction module is used to extract the visual features of the original image;

[0035] A determining module is used to determine fusion parameters based on the visual features; wherein the fusion parameters are used to adapt to the visual style of the original image and to highlight the keywords in the original image;

[0036] A fusion module is used to fuse the target keywords into the original image based on the fusion parameters.

[0037] In a third aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to implement the image generation method described in any of the first aspects when executing the program stored in the memory.

[0038] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements any of the image generation methods described above.

[0039] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the image generation methods described above.

[0040] In this embodiment of the invention, the semantic recognition of the text in the original image determines the user intent, and target keywords representing that intent are extracted. Then, the visual features of the original image are extracted, and based on these features, fusion parameters are determined to match the visual style of the original image and highlight the keywords. Finally, based on these fusion parameters, the target keywords are fused into the original image. Keywords are obtained through semantic analysis of the text in the original image; by extracting the visual features of the original image and matching the corresponding fusion parameters according to the image's visual style, the keywords and the original image are fused together according to the fusion parameters that match the visual style, ultimately generating a target image where the keywords are prominently displayed and fit the style of the original image. When generating target images from original images corresponding to a large number of online resources, this solution eliminates the need for manual processing of each online resource sequentially, improving the efficiency of target image generation. Furthermore, the automatic generation of keywords through semantic analysis eliminates the subjective influence of manual keyword determination. Simultaneously, by setting fusion parameters based on visual style to achieve keyword adaptation and fusion, the solution avoids the visual conflicts and display deviations caused by directly adding keywords, thus improving the accuracy of target image generation. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0042] Figure 1 A flowchart of an image generation method provided in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart illustrating the determination of target keywords in the image generation method provided in this embodiment of the invention;

[0044] Figure 3 A flowchart of a watermark fusion process provided in an embodiment of the present invention;

[0045] Figure 4 A flowchart of another image generation method provided in an embodiment of the present invention;

[0046] Figure 5 This is a structural diagram of an image generation device provided in an embodiment of the present invention;

[0047] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0049] Images are an important medium for attracting user attention and conveying information. For example, in news feed advertising, service platforms can provide users with images displaying advertising content; in online shopping, e-commerce platforms can provide users with images displaying product information; and in social media, users can share images they have taken to share their recent status with other users.

[0050] In real-world scenarios, text with high transparency can be added to images to enhance the message conveyed by the image while minimizing occlusion of its content. For example, in the aforementioned feed advertising scenario, the service platform could be an online shopping platform. For each network resource available to the platform, the original image of that resource displays corresponding advertising content, such as promotional information for a product. Technicians browse the promotional information and manually analyze the text (such as "limited-time discount") most likely to be clicked by users. Then, the text "limited-time discount" is added to the original image of that network resource to obtain the target image. Subsequently, the target image of that network resource is recommended to users, thus achieving the goal of recommending the network resource to them.

[0051] However, this method requires manual processing of each network resource sequentially, which is inefficient. Furthermore, the determination of keywords is easily influenced by human subjectivity, ultimately resulting in low efficiency and accuracy in generating target images.

[0052] To address the aforementioned problems, this invention provides an image generation method applied to an image generation device, such as a server of the aforementioned service platform. The image generation device can pre-generate target images of network resources available from the service platform according to the image generation method provided by this invention. Subsequently, when the push device needs to recommend network resources to a user, it can directly obtain the pre-generated target image and then recommend the target image to the user. Alternatively, when the push device determines that it needs to recommend a network resource to a user, the image generation device can generate the target image of that network resource according to the image generation method provided by this invention and then recommend the real-time generated target image to the user. The push device and the image push device can be the same device or different devices; this invention does not limit this.

[0053] See Figure 1The image generation method provided in this embodiment of the invention includes the following steps: S101, based on the semantics of the text in the original image, identify the usage intent of the original image and extract target keywords representing the usage intent; S102, extract the visual features of the original image; S103, based on the visual features, determine the fusion parameters; wherein, the fusion parameters are used to adapt to the visual style of the original image and make the keywords stand out in the original image; S104, based on the fusion parameters, fuse the target keywords in the original image.

[0054] In this embodiment of the invention, the semantic recognition of the text in the original image determines the user intent, and target keywords representing that intent are extracted. Then, the visual features of the original image are extracted, and based on these features, fusion parameters are determined to match the visual style of the original image and highlight the keywords. Finally, based on these fusion parameters, the target keywords are fused into the original image. Keywords are obtained through semantic analysis of the text in the original image; by extracting the visual features of the original image and matching the corresponding fusion parameters according to the image's visual style, the keywords and the original image are fused together according to the fusion parameters that match the visual style, ultimately generating a target image where the keywords are prominently displayed and fit the style of the original image. When generating target images from original images corresponding to a large number of online resources, this solution eliminates the need for manual processing of each online resource sequentially, improving the efficiency of target image generation. Furthermore, the automatic generation of keywords through semantic analysis eliminates the subjective influence of manual keyword determination. Simultaneously, by setting fusion parameters based on visual style to achieve keyword adaptation and fusion, the solution avoids the visual conflicts and display deviations caused by directly adding keywords, thus improving the accuracy of target image generation.

[0055] This solution adopts a dual-drive decision-making mechanism of "semantic understanding plus visual analysis". Unlike simply adding watermarks based on location or rules, this invention understands the marketing intent of the image text through natural language processing and understands the aesthetic style of the image through visual analysis. This makes the generation of watermarks highly targeted in both content and form, achieving precise enhancement of marketing information and personalized visual presentation. It greatly improves the enhancement effect while ensuring the aesthetic appeal of the original image.

[0056] Regarding step S101, the original image can be an image provided by the provider of the online resource. For example, in the aforementioned information flow advertising scenario, the original image can be an event poster provided by the advertiser; in the aforementioned online shopping scenario, the original image can be a product image provided by the online shopping merchant; in the aforementioned social media scenario, the original image can be a photo provided by the sharing user; if the online resource is a novel, the original image can be a novel cover uploaded by the novel's author; if the online resource is a public welfare activity, the original image can be a promotional image provided by the initiator of the public welfare activity. The actual scenarios may involve far more than these, and this invention does not limit it.

[0057] Text can be displayed in the original image. For example, the aforementioned event poster can display text such as the event start time and event details; the aforementioned product image can display product parameters; and the aforementioned novel cover can display information such as the novel's author and synopsis.

[0058] After acquiring the original image, the image generation device extracts text from the original image using a text extraction algorithm, obtaining the text displayed in the original image as the original text. Such text extraction algorithms can be OCR (Optical Character Recognition), RARE (Robust Scene Text Recognition with Automatic Recognition), etc., and this invention is not limited to these. The extracted original text can be recorded in a text list.

[0059] In real-world scenarios, after an image generation device acquires an image provided by a network resource provider, it can first perform preprocessing on the acquired image, such as size standardization and image noise reduction, and use the preprocessed result as the original image to improve the accuracy of subsequent processing of the original image.

[0060] Image generation devices can perform semantic analysis on the extracted text using natural language processing technology to identify the intended use of the original image, that is, the core communication goals and functional positioning that the image aims to achieve, and extract keywords that can directly represent this intended use. Taking an event poster provided by an advertiser as the original image, the text displayed in the original image may include relevant content such as the event start time, event content description, discount rules, and event theme. The corresponding intended use of the original image is to convey marketing event information to browsing users, highlight the event's discount attributes and attractiveness, so as to attract user attention and guide users to participate in the event. After the semantic analysis model quantifies and scores the marketing value of the text, and matches and filters it with a preset high-value marketing keyword library, words such as "limited-time discount," "free to claim," and "new product launch" that can accurately reflect the core selling points of the event and directly represent the marketing and promotion intentions can be extracted from the original text as keywords.

[0061] In one implementation, the image generation device can achieve the aforementioned semantic analysis and keyword selection functions through a keyword extraction model. For example, the extracted original text can be input into the keyword extraction model to obtain the keywords output by the model. The keyword extraction model can be a pre-trained Natural Language Processing (NLP) model, such as a text classification model based on Bidirectional Encoder Representations from Transformers (BERT), a Generative Pre-trained Transformer (GPT) model, a Large Language Model Meta AI (LLaMA) model, etc. This invention does not limit it to this.

[0062] In another implementation, the image generation device can extract keywords based on a pre-defined high-value keyword library for different scenarios. It then uses semantic matching and weighted calculation to identify the intended use and extract target keywords. Specifically, the device can pre-build corresponding keyword libraries for different application scenarios such as advertising, e-commerce, and public welfare. These libraries contain words with high dissemination value for each scenario and assign corresponding value weights. After extracting the original text, the device performs word segmentation and matches it with the corresponding scenario keyword library. It calculates a comprehensive value score by combining word frequency, contextual relevance, and pre-defined weights. Based on the score, it determines the intended use of the original image and finally selects a pre-defined number of words as target keywords, ranked from highest to lowest score.

[0063] The target keywords may include at least one of the following: the original text contained in the original image, or the text derived from the analysis of the original text.

[0064] In step S102, the image generation device can perform calculations based on each pixel in the original image to obtain the color, brightness, and texture attributes of the original image, thereby obtaining the visual features of the original image.

[0065] For example, the image generation device first converts the original image to a color space, separating independent luminance and color channels. For the luminance channel, the device iterates through all pixels in the image, reading the luminance value of each pixel one by one. The sum of all pixel luminance values ​​is then divided by the total number of pixels in the image to calculate the average brightness of the entire image. For the color channel, the device performs clustering operations on the hue and saturation values ​​of all pixels, calculating the coverage area ratio of each color category in the image. These color groups are then sorted from highest to lowest ratio, and the top-ranked color groups are selected as the dominant color tones of the image. Simultaneously, the hue, saturation, and brightness parameters corresponding to each dominant color group are recorded. The device can also compare the color differences between adjacent pixels row by row and column by column to calculate the drastic degree of color changes in the image, obtaining a quantitative value for the image's texture complexity.

[0066] After completing the basic visual parameter calculations, the image generation device inputs the original image into a pre-trained image style classification model. The model performs multi-layer convolution operations on the input image to extract deep visual features, obtaining a feature vector representing the overall style of the image. This feature vector is then compared with pre-stored style feature benchmarks such as minimalist, retro, cartoon, and realistic styles to calculate similarity. Finally, the style category with the highest similarity is output as the style label for the image. Subsequently, the device inputs the original image into a target detection model. The model generates candidate detection boxes of different sizes within the image area, performs category determination and confidence calculation on the image content within each candidate box, and filters out detection results with confidence scores meeting a preset threshold. This identifies core subject areas such as people, goods, and logos in the image and records the boundary coordinates, width, and height dimensions of each subject area.

[0067] Regarding step S103, the image generation device can determine the fusion parameters based on the extracted visual features. Among these, "adapting to the visual style of the original image" means that the fusion parameters have the function of coordinating the keywords and their carrying elements with the color expression, texture characteristics, and overall visual tone of the original image. This ensures that after the keywords and the original image are fused, the overall visual appearance of the image is harmonious and unified, without producing a jarring sense of disjointedness, and without compromising the original visual integrity and aesthetics of the original image. "Making the keywords stand out in the original image" means that the fusion parameters have the function of enhancing the visual distinction between the keywords and the background of the original image. This strengthens the visual prominence of the keywords, allowing users to quickly grasp the core information carried by the keywords when browsing the image, ensuring the efficiency and effectiveness of conveying the core information.

[0068] Taking marketing images as an example, to adapt the visual style of the original image, if the original image is a retro-style beauty product poster with a warm brown main color and film grain texture, the corresponding blending parameters will set the watermark base to a warm brown gradient color, overlay a low-intensity grain noise texture, and use a serif font for the keywords, with the font color taken from the darker shade of the same color family as the original image. If the original image is a minimalist novel cover with a large area of ​​solid white space, the corresponding blending parameters will set the watermark base to a solid color semi-transparent style, without adding any additional texture, and use a sans-serif font for the keywords, with the font color matching the darker value of the main color family of the original image.

[0069] To make keywords stand out in the original image, if the overall average brightness of the original image is low and the image is mainly dark, the corresponding blending parameters will select a high-brightness light color as the keyword font color, and at the same time match a light-colored semi-transparent background below the text. If the original image has a high degree of texture complexity and the background elements are fragmented and rich, the corresponding blending parameters will configure a solid color background for the keywords, and at the same time enlarge the font size of the keywords proportionally.

[0070] The visual features can include at least one of the following: basic color brightness features, texture features, style category features, and subject area layout features. Basic color brightness features can encompass the overall average brightness value of the image, the hue, saturation, and brightness parameters corresponding to the main color tone, and quantitative data such as the average color saturation of the entire image. Based on these features, fusion parameters such as the watermark base color and transparency can be determined. Texture features can include quantitative values ​​of the image's texture complexity and image granularity parameters. Based on these features, fusion parameters such as the texture style and transparency threshold of the watermark base can be determined. Style category features are style labels representing the overall visual tone of the image, such as classifications like minimalist, retro, cartoon, and realistic. Based on these features, fusion parameters such as the corresponding font type, base texture, and color matching rules can be determined. Subject area layout features include the boundary coordinates of the core subject within the image, the area's width and height dimensions, and the distribution information of non-subject blank areas within the image. Based on these features, the display range of keywords can be determined.

[0071] The image generation device can determine the corresponding fusion parameters based on different items in the visual features. If there is only one visual feature, the corresponding fusion parameter is determined based on that visual feature; if there are multiple visual features, the corresponding fusion parameter is determined based on each visual feature. Alternatively, the corresponding fusion parameter can be determined for only some of the multiple visual features as needed.

[0072] In step S104, the image generation device can integrate the target keywords into the original image based on the aforementioned determined fusion parameters, thereby completing the fusion processing of the keywords and the original image.

[0073] In one implementation, the image generation device first generates a watermark base layer. The device retrieves the base color parameters, transparency parameters, and texture parameters from the blending parameters, generates a base layer of the same size as the original image, fills the layer with a background color of the corresponding hue, saturation, and brightness, adjusts the overall opacity of the layer to the set value, and finally overlays a texture effect of the corresponding type on the base layer according to the texture parameters, so that the visual attributes of the base layer match the visual style of the original image.

[0074] After the base layer is created, the device renders the target keywords to the designated area of ​​the base layer based on the font parameters, text color parameters, and layout position parameters in the blending parameters. The device selects a matching font style and font size to render the text, sets the text color to the corresponding parameter value, and, according to the layout constraints in the blending parameters, arranges the text as a whole on the base position corresponding to the non-subject blank area of ​​the original image, avoiding the text from obscuring the core subject in the original image.

[0075] The device can also add target keywords to the original image by generating independent keyword images and then overlaying them. Specifically, the device can first generate a keyword image carrying the target keyword, and then overlay the keyword image with the original image. To reduce the probability of occlusion of existing content such as text and patterns in the original image, the device can determine the text area occupied by the text and the pattern area occupied by the pattern in the original image based on a graphic layout algorithm, select an image area that does not intersect with the above two types of areas, and use the size of the area as the adaptation size of the keyword image. The device can generate a keyword image displaying the target keyword according to a preset layout format, such as center alignment, distributed alignment, and preset spacing from the image edges. To ensure the visual prominence of the keyword, the device can obtain the contrast color of the main color tone of the original image, and use this contrast color as the font color to adjust the display effect of the target keyword in the keyword image to obtain the final keyword image. This invention does not limit this.

[0076] After the text rendering is complete, the device will overlay the watermark base layer carrying the target keywords onto the original image. The device can first feather the edges of the base layer to soften the boundary transition effect of the layer, and then overlay the base layer onto the corresponding layer level of the original image according to the set blending mode, so that the base layer and the original image are naturally connected, and finally output the target image with keyword fusion.

[0077] After acquiring the target image, the image generating device can store it in an image library. Later, when the target image needs to be pushed to the user, the push device can retrieve it from the image library. Alternatively, the image generating device can send the target image to the push device immediately after acquiring it; subsequently, the push device directly determines which target image to push to the user from the target images it has received. This invention does not limit the scope of the invention to this method.

[0078] In one possible implementation, see Figure 2 In the image generation method provided in this embodiment of the invention, step S101 can be implemented as follows: S1011, extract text from the original image to obtain the original text; S1012, perform semantic analysis based on the original text to identify the usage intent of the original image and extract candidate keywords that represent the usage intent; and obtain the predicted click probability corresponding to the candidate keyword based on the predicted click probability of the image displaying each candidate keyword; S1013, determine at least one candidate keyword with the highest predicted click probability from each candidate keyword as the target keyword representing the usage intent.

[0079] In this embodiment of the invention, the solution eliminates the need for manual text recognition, intent judgment, and keyword filtering for each network resource sequentially. The entire process can be completed through automation, improving the overall efficiency of keyword extraction. Furthermore, semantic analysis automatically identifies the intended use of the original image and generates candidate keywords, eliminating subjective biases caused by manual judgment of image intent and keyword filtering, ensuring the alignment between keywords and image intended use. Simultaneously, each candidate keyword is matched with a corresponding predicted click probability. The attractiveness of the keywords is quantitatively evaluated based on user click behavior data, and the candidate keyword with the highest predicted click probability is selected as the final target keyword. This allows for the selection of core keywords with stronger user appeal, further improving the accuracy of target keyword selection and actual usage intent.

[0080] In step S1011, the image generation device can use a text recognition algorithm to detect and recognize text in the original image, extract all the text content carried in the image, and obtain the original text, providing a basic processing object for subsequent semantic analysis, intent recognition and keyword selection.

[0081] For example, the image generation device can use text extraction algorithms such as OCR and RARE to implement this step. The device first performs a full scan of the original image, detecting and locating all regions containing text, distinguishing between text regions and background pattern regions. Then, it recognizes each character within each text region, simultaneously restoring the line and sentence order to ensure the recognition results conform to normal reading logic. Finally, all the recognized text content is integrated into a complete text set, which is the original text. For original images that have undergone preprocessing such as size standardization and image noise reduction, the device can directly perform text extraction based on the preprocessing results, thereby reducing the interference of image noise and size differences on recognition accuracy. The extracted original text can be stored as a text list for subsequent semantic analysis processing.

[0082] In step S1012, the image generation device performs semantic analysis on the original text based on natural language processing technology. On the one hand, it identifies the usage intent corresponding to the original image and extracts candidate keywords that match the intent. On the other hand, it simultaneously derives the predicted click probability corresponding to each candidate keyword, providing a quantitative evaluation basis for the subsequent selection of target keywords.

[0083] Among them, the candidate keywords may include at least one of the following: the original text contained in the original image, the text derived from the analysis of the original text, and the image generation device may select the target keyword from the candidate keywords.

[0084] For example, the image generation device inputs the original text into a pre-trained keyword extraction model. The model first performs semantic encoding and contextual understanding on the original text, and determines the intended use of the original image based on the semantic meaning of the text content. For instance, if the original text contains content related to activity rules or discounts, the intended use of the original image is determined to be a marketing campaign promotion. Based on the identified intent, the model extracts multiple words associated with the intended use from the original text, forming a set of candidate keywords. These candidate keywords are words that can reflect the core communication goal of the image from different dimensions.

[0085] Meanwhile, the keyword extraction model, based on the correlation between keywords and click probabilities learned during the training phase, predicts the likelihood of user clicks for each candidate keyword and outputs the predicted click probability for each candidate keyword. This predicted click probability is learned from user click data of similar keywords in historical scenarios; a higher value indicates that the image containing the candidate keyword is more likely to attract user clicks, thus intuitively quantifying the actual dissemination appeal of the candidate keywords.

[0086] For example, the keyword extraction model can be trained based on sample text, the sample keywords corresponding to the sample text, and the sample click probability corresponding to each sample keyword. This enables the model to assess the marketing appeal of keywords and match the intended use of images based on text semantics. Taking the sample text as the text in an advertising campaign poster as an example, the sample text can be taken from the text identified in the campaign posters provided by the advertiser in the past, that is, the relevant text representing the content of the past campaign. The sample keywords are the keywords carried in the target images pushed in the past, such as marketing terms like "free to receive," "discount," "new product launch," "limited time," and "buy one get one free." In a real-world scenario, the push device can recommend the same historical target image to multiple users, and the same user can also browse multiple different historical target images. After browsing the images, users can jump to the corresponding campaign interface by clicking. The image generation device can count the number of user clicks on each historical target image to obtain the click frequency of the corresponding image. This click frequency can directly reflect the attractiveness of the sample keywords in the image to the user. The higher the click frequency, the higher the probability that the corresponding keyword will guide the user to click.

[0087] Based on the click frequencies corresponding to each historical target image, the click frequencies corresponding to each sample keyword can be further derived. If multiple historical target images contain the same sample keyword, the sum, mean, or maximum value of the click frequencies of these images can be obtained as the click frequency corresponding to that sample keyword. Subsequently, the image generation device can determine the sample click probability corresponding to each sample keyword according to the magnitude of the click frequency of each sample keyword. The sample click probability is positively correlated with the click frequency corresponding to the keyword. Specifically, the click frequency can be normalized to the interval [0, 1], and the normalization result can be used as the sample click probability of the corresponding sample keyword.

[0088] To reduce computational load and ensure model learning effectiveness, the image generation device can first select the images with the highest click frequency from all historical target images. It then calculates the sample click probability based solely on the sample keywords corresponding to these selected images, and uses this data for model training. By selecting high-click-frequency sample data for training, the model can focus on learning the correlation between highly attractive keywords and click probabilities, ensuring that the keywords output by the subsequent model have stronger user appeal. After preparing the sample data, the image generation device inputs the sample text, sample keywords, and corresponding sample click probabilities into the keyword extraction model, guiding the model to learn the mapping relationship between text semantics, keywords, and click probabilities, thus completing the model training process. In the aforementioned information flow advertising scenario, the keyword and corresponding weight mapping relationship learned by the trained model corresponds to the aforementioned high-value marketing keyword library, where the sample click probability can also be referred to as a weight or marketing attractiveness score.

[0089] In practical applications, image generation devices can also input prohibited keywords into the model and provide prompts to prevent the model from outputting such words, thus fine-tuning the model and further adapting it to business specifications. For example, in the context of news feed ads, expressions such as "super" and "most" that do not comply with advertising regulations can be set as prohibited words to prevent the model from outputting illegal content.

[0090] Once the model is trained, it can be used for keyword extraction from the original text. The image generation device inputs the extracted original text, prompts indicating keywords for the model, and prompts indicating the predicted click probabilities for each keyword into the keyword extraction model. The model outputs all candidate keywords identified from the original text, along with the predicted click probability for each candidate keyword. This predicted click probability is derived by the model based on learned relationships and objectively represents the probability that an image containing the corresponding keyword will be clicked by a user. In some embodiments, the image generation device can periodically iterate and update the keyword extraction model according to the above training logic, optimizing the model's performance based on continuously generated user click data from real-world scenarios to continuously improve the accuracy of keyword extraction.

[0091] In another implementation, the image generation device can input the original image into a multimodal large language model, extract the original text from the original image through the image recognition capability of the multimodal large language model, identify each original text through the text analysis capability of the multimodal large language model, analyze and summarize the candidate keywords from each original text, and predict the click probability of each candidate keyword.

[0092] For step S1013, the image generation device sorts and filters all candidate keywords using the predicted click probability as the filtering index, and selects at least one candidate keyword with the highest predicted click probability as the target keyword representing the intended use of the original image, thus completing the final output of the keyword extraction step.

[0093] For example, the image generation device first sorts all candidate keywords from highest to lowest according to their predicted click probabilities, and then determines the final target keywords based on preset filtering rules. The filtering rules may include constraints such as the maximum number of keywords a single image can carry and the maximum total number of characters. For instance, technicians can preset the maximum number of keywords that can be displayed for different image sizes. When the image size is 1920*1080, the total number of keywords displayed in the image can be set to no more than 4 characters. The image generation device first matches the maximum number of characters corresponding to the current original image size, and then selects candidate keywords in descending order of predicted click probabilities until the total number of characters of the selected keywords does not exceed the maximum number of characters. The final 1 to 3 keywords are the target keywords representing the intended use of the original image.

[0094] In one possible implementation, step S1011 includes: A1, if there is a preset keyword with a similarity greater than a preset threshold to each candidate keyword, then the preset click probability corresponding to the preset keyword is used as the predicted click probability corresponding to the candidate keyword; wherein, the preset click probability corresponding to each preset keyword represents the click frequency of the image displaying the preset keyword; A2, if there is no preset keyword, then the candidate click probability corresponding to the candidate keyword output by the prediction model is obtained as the predicted click probability of the candidate keyword; wherein, the prediction model is trained based on sample text, sample keywords in the sample text, and the actual click probability corresponding to each sample keyword.

[0095] In this embodiment of the invention, for candidate keywords whose similarity to preset keywords meets the threshold requirement, the corresponding preset click probability can be directly retrieved as the result, eliminating the need for repeated model inference calculations. This allows for rapid determination of the predicted click probability based on historical statistical data of real image click frequencies. For candidate words without matching preset keywords, a pre-trained prediction model outputs the corresponding candidate click probability, covering emerging expressions and personalized words not included in the preset thesaurus, avoiding the problem of missing probability acquisition due to the limitation of the thesaurus. These two paths complement each other, providing reliable predicted click probability support for all candidate keywords, providing accurate quantitative basis for subsequent target keyword selection, and thus improving the overall efficiency of target keyword selection and actual dissemination effect.

[0096] Regarding step A1, the preset keywords are standardized terms pre-collected in a preset keyword library, containing historical business data and user-added terms as needed. These terms correspond to high-frequency core expressions in various image usage scenarios and can serve as a benchmark for semantic matching, providing reusable historical click data support for semantically similar candidate keywords. Taking the information flow advertising scenario as an example, the preset keywords can include universally validated marketing terms such as "limited-time discount," "free to claim," "new product launch," and "buy one get one free," covering the core selling points of mainstream activities.

[0097] The preset click probability for each preset keyword is obtained by statistically analyzing the click frequency of images displaying that keyword within a historical period. The image generation device aggregates the total exposure count and total user click count of all images carrying that preset keyword within a historical period. The click frequency of the corresponding image is obtained by calculating the ratio of click count to exposure count. This click frequency is then normalized to the range [0, 1] to finally obtain the preset click probability for that preset keyword. This value objectively reflects the level of user click attraction of the preset keyword in the actual deployment scenario. For example, for the preset keyword "free to claim", the device statistically analyzes that all images displaying this keyword have accumulated one million exposures and one hundred thousand user clicks in history, corresponding to a click frequency of 10%. After normalization, 0.1 is taken as the preset click probability for that preset keyword.

[0098] After obtaining candidate keywords, the image generation device calculates the similarity between each candidate keyword and each preset keyword in the preset keyword library, and compares the calculation result with a preset threshold. If the similarity between a candidate keyword and a preset keyword in the library is greater than the preset threshold, it is determined that the two are semantically highly similar and have the same level of appeal to users. The device then directly retrieves the preset click probability corresponding to that preset keyword and uses it as the predicted click probability for the current candidate keyword. For example, if the candidate keyword is "limited-time offer," and its similarity to the preset keyword "limited-time discount" is higher than the preset threshold after semantic calculation, the device directly reuses the preset click probability corresponding to "limited-time discount" as the predicted click probability for the candidate keyword "limited-time offer."

[0099] For step A2, the prediction model is a pre-trained natural language processing model. Its training data includes sample text, corresponding sample keywords, and the actual click probability for each sample keyword. The sample text and keywords are taken from the text content and keywords corresponding to various historically displayed images. The actual click probability is obtained by statistically analyzing the actual exposure and click volume of images displaying the corresponding sample keyword within a historical period, consistent with the statistical logic of the preset click probability, thus accurately reflecting the user click appeal of the keyword in real-world scenarios. During training, the model continuously learns the mapping relationship between text semantics, keyword semantics, and actual click probability, ultimately possessing the ability to output a predicted click probability value for any keyword. For example, this prediction model can be the keyword extraction model in the aforementioned embodiments.

[0100] When a candidate keyword cannot be matched with a preset keyword in the preset keyword library with a similarity greater than a preset threshold, the image generation device inputs the candidate keyword along with its original text context into the prediction model. The model infers based on the learned semantics and click probability relationship and outputs the candidate click probability corresponding to the candidate keyword. The device directly uses the candidate click probability as the predicted click probability corresponding to the current candidate keyword.

[0101] Taking the information flow advertising scenario as an example, if the candidate keyword is "new user exclusive price", and no corresponding preset keyword matching the threshold is found in the preset keyword library after semantic matching, the device will input the candidate keyword and the original activity text into the prediction model. The model will combine the click data patterns of similar new user discount keywords in history to infer and output the corresponding candidate click probability.

[0102] In one possible implementation, see Figure 3Step S104 includes: S1041, generating a watermark base based on the first parameter in the fusion parameters that describes the overall style of the original image; S1042, adjusting the target keyword according to the second parameter in the fusion parameters that describes the visual form of the keyword in the image; S1043, rendering the adjusted target keyword on the watermark base to obtain a fused watermark carrying the target keyword; S1044, fusing the fused watermark in the original image based on the third parameter in the fusion parameters that describes the position of the keyword.

[0103] In this embodiment of the invention, a watermark base is generated based on a first parameter describing the overall style of the original image. This allows the base's color, texture, transparency, and other attributes to naturally adapt to the visual style of the original image, providing a unified carrier for keywords and avoiding visual confusion and disjointed visual experience caused by keywords being directly superimposed on complex backgrounds. The target keywords are adjusted according to a second parameter describing their visual form, allowing for simultaneous adaptation to the image style and highlighting requirements in terms of font, size, and color. This ensures stylistic consistency while increasing the visual contrast between the keywords and the carrier base, enhancing keyword recognizability. The adjusted target keywords are first rendered on the watermark base to obtain a complete fused watermark. Keywords and the carrier base can be adjusted and reused as independent modules without modifying the original image pixel by pixel, reducing computational complexity and facilitating flexible iteration of watermark styles. Finally, the fused watermark is integrated into the original image based on a third parameter describing the keyword's position. This precisely avoids the core subject area within the image, adapting to the compositional logic of the original image and ensuring prominent keyword display without compromising the integrity of the original image's core content. The overall solution can adapt to original images with different styles and compositions, effectively improving the visual harmony and information transmission effect after keyword fusion, and ensuring the quality of the target image.

[0104] This solution employs an adaptive visual style watermark base generation technique: dynamically generating the color, transparency, and texture of the watermark base based on the main color tone, brightness, and style of the original image. Its beneficial effect is that the generated watermark visually appears as an extension of the original image design, rather than a rigid addition, significantly enhancing the overall visual harmony and professionalism of the synthesized target image.

[0105] Regarding step S1041, the overall style of the original image is the consistent visual characteristics presented by the original image in dimensions such as color tone, texture expression, and design type. It is a general representation of the overall visual tone of the image and directly determines the overall visual tendency of the picture. For example, retro, minimalist, cartoon, and realistic are all different overall style types, corresponding to different color usage logic and texture expression forms.

[0106] The first parameter describing the overall style of the original image is a set of configuration parameters formed by quantifying the above style features. It may include the main color tone parameter, texture feature parameter, style classification label, overall brightness parameter, etc. of the original image. It is the core basis for controlling the appearance attributes of the watermark base and is used to ensure that the visual tone of the generated base is consistent with that of the original image.

[0107] The watermark base is the underlying graphic layer that carries the target keywords. Its core functions are twofold: first, it acts as a style transition layer, inheriting the stylistic characteristics of the original image and mitigating the visual dissonance caused by keywords being directly superimposed on the original image, ensuring the overall harmony of the image; second, it serves as a unified background for text display, differentiating the keywords from the complex background of the original image, reducing the interference of background textures and colors on text readability, and improving the readability and visual recognizability of the keywords. In practical applications, the watermark base can take various forms, such as solid color semi-transparent blocks, gradient color bars, and backgrounds with subtle textures, and can be flexibly generated according to the stylistic requirements of different images.

[0108] For example, the image generation device first reads the hue, saturation, and brightness values ​​of the primary color contained in the first parameter, generates an initial layer with the same size as the original image, and fills the initial layer with the base color corresponding to the parameters. Then, based on the style category label and texture feature parameters in the first parameter, the device overlays a corresponding type of texture effect onto the initial layer. When the first parameter corresponds to a retro style, a low-intensity film grain texture is overlaid on the layer; when it corresponds to a minimalist style, no additional texture is added to the layer; and when it corresponds to a cartoon style, a soft color gradient transition effect is added to the layer. After completing the color and texture configuration, the device adjusts the overall transparency of the layer according to the overall brightness parameter in the first parameter. The higher the overall brightness of the original image, the higher the base transparency is appropriately increased; the lower the overall brightness of the original image, the lower the base transparency is appropriately decreased, ultimately generating a watermark base that matches the overall style of the original image.

[0109] Regarding step S1042, the visual form of keywords in an image refers to all the visible appearance attributes of the target keywords when presented in text form. This determines the style fit and visual recognizability of the text within the image. Specifically, the visual form of keywords in an image can include at least one of the following dimensions: font style, font size, font color, and font weight. These are core elements affecting the style adaptation effect and information transmission efficiency of keywords. Different visual forms convey different visual feelings. For example, serif fonts tend to be retro and formal, while sans-serif fonts tend to be simple and modern. Text with high-saturation colors has a stronger visual impact, and small-sized text has a weaker presence.

[0110] The second parameter, describing the visual appearance of the keyword, is a subset of parameters in the fusion parameter set used to control the appearance of the keyword text. Its core function is to transform the dual requirements of style adaptation and highlighting the display into directly executable, quantifiable configuration items. This sets a clear appearance benchmark for the rendering of the target keyword, ensuring that the adjusted keyword is both consistent with the visual tone of the watermark base and the original image, while also possessing sufficient visual distinctiveness. The second parameter typically includes sub-items such as font type parameters, font size parameters, font color parameters, and font weight parameters. Each parameter corresponds to controlling one of the keyword's appearance attributes, collectively achieving precise adjustment of the keyword's visual appearance.

[0111] For example, for the aforementioned retro-style film-like product image, the corresponding second parameter would be configured with a serif font, a medium font size proportional to the image size, and a dark brown font color that contrasts with the watermark base. After reading this set of second parameters, the image generation device would switch the font of the target keyword to the specified serif font, adjust the font size to the set size, and change the font color to the corresponding dark brown, thus completing the adjustment of the visual form of the target keyword. This would ensure that the text style of the keyword matches the retro tone, while also providing sufficient contrast with the base color to guarantee readability. If the original image is a minimalist solid-color cover, the corresponding second parameter would be configured with a sans-serif font, a larger font size, and a pure white font color. The adjusted keyword visual presentation would be clean and crisp, matching the minimalist style of the image while clearly standing out on a semi-transparent base.

[0112] Regarding step S1043, during rendering, the image generation device can convert the target keywords in vector form into pixelated graphic information and draw them onto the watermark base layer according to specified layout rules. When merging the watermark, the keywords can be bound to the watermark base as a single operable object. When merging with the original image later, operations such as position adjustment, transparency adjustment, and edge softening can be performed directly on this object without separately managing the base and the text.

[0113] For example, during the rendering process, the image generation device first calculates the drawing position of the target keyword on the base according to the size of the watermark base and preset layout rules. For instance, if the keyword is set to be horizontally and vertically centered within the base, and the edges of the text and the base maintain a preset proportion of white space, the device calculates the coordinates of the base layer corresponding to the upper left corner of the text. After determining the position, the device converts the target keyword, whose font, size, and color have been adjusted, from vector text data into pixel data with the same resolution as the base layer, and draws it pixel by pixel onto the watermark base layer according to the calculated coordinates, so that the text pixels and base pixels are merged into the same layer. After drawing, the device can perform anti-aliasing smoothing on the text edges to weaken the jaggedness of the text edges and make the transition between the text and the base more natural. Finally, the complete layer carrying the target keyword is the fused watermark, which can be directly used for subsequent overlay and fusion with the original image.

[0114] In one possible implementation, the first parameter includes: the main color tone of the original image, the overall brightness information of the original image, and the style type of the original image; step S1041 includes: B1, determining the color and transparency of the watermark base based on the main color tone and overall brightness information included in the first parameter of the fusion parameters; B2, generating the watermark base to be processed according to the determined color and transparency; B3, determining the texture corresponding to the style type in the fusion parameters; B4, adding the determined texture to the watermark base to be processed to obtain the desired watermark base.

[0115] In this embodiment of the invention, the color and transparency of the watermark base are determined based on the main color tone and overall brightness information of the original image. This ensures that the base color is consistent with the main color scheme of the original image. At the same time, the transparency is dynamically adjusted according to the brightness of the image. While providing a stable background for the keywords, this avoids the base from being too abrupt and disrupting the visual balance of the original image, thus ensuring the overall harmony of the image.

[0116] In addition, the watermark base to be processed is generated first according to the determined color and transparency. The core attributes of the base can be standardized and output to form a unified basic carrier, providing a standard layer for subsequent texture overlay and simplifying the subsequent texture adjustment process.

[0117] For step B1, the first parameter is a set of parameters describing the overall style of the original image, including the main color tone, overall brightness information and style type of the original image; among them, the main color tone and overall brightness information are the objects called in this step, which respectively control the color and transparency attributes of the watermark base, while the style type is used in the subsequent texture matching process.

[0118] The dominant color tone of the original image is the core color system that dominates the overall color perception of the original image. It is quantified by the corresponding hue, saturation, and brightness values ​​and is the core element that determines the overall color tone of the image. Its function is to serve as the color reference for the watermark base, ensuring that the base color and the original image are in the same color system and avoiding visual disjointedness caused by color system conflicts. During the process, the image generation device extracts the hue and saturation parameters of the dominant color tone, and after fine-tuning the brightness and reducing the saturation, determines it as the fill color of the watermark base, so that the base color and the main color of the original image are consistent and do not overshadow the visual center of the image.

[0119] The overall brightness information of the original image is a quantitative representation of the average brightness and darkness of the entire image, usually expressed as the average brightness of all pixels. It can intuitively reflect the overall visual tendency of the image to be too bright or too dark. Its function is to provide an adaptation basis for the transparency setting of the watermark base, balancing the base's effect on text contrast with its degree of obstruction of the original image. During execution, the image generation device maps the overall brightness value to a preset transparency range. The higher the overall brightness of the image, the higher the corresponding base transparency setting, reducing the base's heaviness in a bright image; the lower the overall brightness of the image, the lower the corresponding base transparency setting, strengthening the base's ability to highlight the text and avoiding confusion between keywords and a dark background.

[0120] For example, if the original image is a bright, minimalist style poster of daily necessities, with a main color of off-white and high overall brightness, in step B1, the device, based on the off-white main color, fine-tunes and determines the watermark base to be a light beige-gray of the same color family; for the high brightness, the base transparency is set to 70%, making the base soft and natural in the bright image, while also slightly highlighting the text. If the original image is a dark-toned night scene style event poster, with a main color of navy blue and low overall brightness, the image generation device determines the base to be a low-saturation dark gray-blue based on the navy blue main color, and sets the base transparency to 40% based on the low brightness, giving the base sufficient coverage to support the light-colored keywords and ensure the text is clear and readable.

[0121] In step B2, the watermark base to be processed is an intermediate form in the watermark base generation process. Only basic color filling and global transparency configuration have been completed; style texture effects have not yet been overlaid. During the generation operation, the image generation device first reads the size and resolution parameters of the original image and creates a blank transparent layer with the same resolution as the original image. Then, the image generation device uses the base color determined in step B1 as the fill color to fill the blank layer with a solid color, giving the entire layer a uniform basic color. After filling, the device applies the transparency value determined in step B1 to this layer, adjusting the layer's global opacity to give the solid color layer a corresponding degree of semi-transparency. After these three steps—size matching, color filling, and transparency adjustment—the resulting solid color semi-transparent layer is the watermark base to be processed.

[0122] Regarding step B3, the style type of the original image can be used to summarize the visual style attributes of the image. Common categories include retro, minimalist, cartoon, realistic, technological, and traditional Chinese style. Texture refers to the subtle textural effects that can be superimposed on the watermark base. Its core function is to ensure that the surface texture of the base is consistent with the texture of the original image, further weakening the abruptness of the superposition between the watermark base and the original image, and improving the overall style integration. Textures typically include configuration parameters such as texture type, grain size, and superposition intensity; different textures are suitable for different style scenarios.

[0123] When performing the determination operation, the image generation device pre-maintains a mapping rule library of style types and texture styles, with a suitable texture scheme configured for each style type. After reading the style type identifier in the first parameter, the image generation device searches the mapping rule library and retrieves the texture style and matching parameters that correspond one-to-one with that style type, thus completing the determination of the corresponding texture.

[0124] For example, following the aforementioned dark-toned retro beauty poster scenario, its style type is retro, and the mapping rule library corresponds to a film grain texture in the retro style, with fine grains and low overlay intensity. When executing step B3, the device recognizes the style type as retro and matches and determines a fine-grained, low-intensity film grain texture. This texture is consistent with the retro film texture of the original image, and after overlay, it allows the texture of the watermark base to be naturally unified with the original image. If the original image is a minimalist daily necessities poster, the style type is minimalist, and the corresponding matching texture is a very light frosted texture or no texture, thus matching the clean and crisp texture of the minimalist image; if the original image is a tech-themed digital product poster, the style type is tech-themed, then a fine, low-opaque linear grid texture is matched to adapt to the tech tone of the image.

[0125] In step B4, the image generation device adds the determined texture to the watermark base to be processed, giving the base a texture effect that matches the style of the original image, ultimately resulting in a complete watermark base. The completed watermark base possesses suitable color, transparency, and style texture, and can be directly used as the underlying substrate to carry the target keywords, maintaining harmony and unity with the overall visual style of the original image.

[0126] Based on the generation of the watermark base described above, the size, transparency, and subsequent fusion method of the target watermark base can be further refined to adapt to the original image requirements of different scenarios.

[0127] The target watermark base is a semi-transparent layer that covers the original image, and its watermark size is usually no larger than the image size of the original image. The transparency of the target watermark base directly determines the visibility of the target keywords in the subsequent target image; in order to balance the degree of occlusion of the original image content and the recognizability of the keywords, the transparency (alpha) of the target watermark base can be set within the transparency range of [0.1, 0.3] in actual scenarios.

[0128] The image generation device can generate a layer that conforms to the above transparency range according to the image size as the target watermark base. A keyword image displaying the target keyword is added to the target watermark base to obtain the target watermark image. The target watermark is then added to the original image, either by direct overlay or by feathering the edges of the target watermark to make the fusion effect more natural, and finally the target image is obtained.

[0129] The target image is generated by overlaying the target watermark image without modifying the original image itself, thus reducing the intrusion into the original image. Even in scenarios where the original image provider does not allow direct modification of the original image, keyword fusion can still be completed to obtain the corresponding target image, expanding the applicability of the image generation method and adapting to the diverse needs of actual business.

[0130] In one implementation, the size of the target watermark base can be consistent with the image size of the original image. After obtaining the target watermark image, it is overlaid on top of the original image, and the pixels of the target watermark image and the original image are aligned one by one, which can ensure that the target keyword is fully displayed within the range of the original image and improve the generation quality of the target image.

[0131] In another implementation, step S1041 can also determine the adaptation of irregularly shaped images based on aspect ratio, specifically including the following steps:

[0132] If the original aspect ratio of the original image is greater than the first aspect ratio threshold, a watermark base of the first size is generated as the target watermark base; if the original aspect ratio is less than the second aspect ratio threshold, a watermark base of the second size is generated as the target watermark base.

[0133] The aspect ratios of the first size and the second size are both between the first aspect ratio threshold and the second aspect ratio threshold, and both sizes are smaller than the original size of the original image; the first aspect ratio threshold is greater than the second aspect ratio threshold.

[0134] Accordingly, the aforementioned step S1043 can be adapted to: adding the target watermark image to the center position of the original image to obtain the target image.

[0135] There is no strict limitation on the execution order of this step. The aspect ratio determination and base generation can be performed first, followed by the keyword extraction and filtering steps; or the keyword processing can be completed first, followed by the generation of the watermark base of the corresponding size; or the two can be performed in parallel.

[0136] In real-world scenarios, original images come in various sizes. For example, in news feed ads, there are often ads with an original aspect ratio of 16:3. Since users' visual attention is usually focused on the center of the image, if a watermark base of the same size is generated for an image with an extreme aspect ratio, the target keywords will be scattered along with the base, resulting in fragmented information delivery and weakening the enhancement effect on the core information of the original image.

[0137] Therefore, original images with an aspect ratio greater than the first aspect ratio threshold or less than the second aspect ratio threshold are defined as irregular images; watermark bases with smaller size and reasonable aspect ratios are generated for irregular images, which can concentrate the display range of keywords.

[0138] The aspect ratio threshold can be determined through business statistics: technical personnel can pre-calculate the aspect ratio specifications of common original images, measure the time users spend browsing keywords under different aspect ratios, and classify extreme aspect ratio images with high browsing time as irregular images; the minimum value among irregular aspect ratios greater than 1 is the first aspect ratio threshold, and the maximum value among irregular aspect ratios less than 1 is the second aspect ratio threshold. Simultaneously, corresponding watermark base sizes can be preset for various irregular images to ensure that the base size is smaller than the original image and the aspect ratio is within a reasonable range.

[0139] In actual execution, the image generation device first obtains the original dimensions of the original image and determines whether its aspect ratio exceeds a threshold range or whether it is an irregularly shaped image. If it is a regular image, a watermark base of the same size is generated; if it is an irregularly shaped image, a watermark base of the corresponding size is generated according to preset rules. Finally, the target watermark image carrying the keywords is superimposed on the center of the original image, so that the keywords are concentrated in the user's visual focus area, improving information transmission efficiency and reinforcement effect.

[0140] In some embodiments, keyword readability can be further improved by brightness inverse matching, i.e., when the image brightness of the original image is greater than a preset brightness threshold, the brightness value of the target watermark base is less than the preset brightness threshold; when the image brightness of the original image is less than the preset brightness threshold, the brightness value of the target watermark base is greater than the preset brightness threshold.

[0141] In this scheme, the brightness value of the target watermark substrate directly affects the display brightness of the keywords. If the brightness of the original image and the keywords is similar, it will lead to blurred keyword boundaries and reduced readability. Therefore, the image generation device can first calculate the average brightness of all pixels in the original image as the image brightness, and use the median brightness value as the preset brightness threshold. For example, when the brightness range is [0, 255], the preset brightness threshold can be set to 128. Images with brightness below the threshold are considered dark images, and those above the threshold are considered bright images. A high-brightness bright substrate is generated for dark images, and a low-brightness dark substrate is generated for bright images, thus strengthening the keyword boundaries through brightness contrast.

[0142] In practice, a first brightness value corresponding to the dark image and a second brightness value corresponding to the bright image can be preset, which are fixed values ​​higher and lower than preset brightness thresholds, respectively. Alternatively, the brightness difference can be calculated, that is, the difference between the maximum brightness value and the brightness of the original image is used as the base brightness value. By using bright images with dark bases and dark images with bright bases, the boundary clarity and readability of target keywords can be significantly improved.

[0143] To further enhance the naturalness of the blend, the target watermark base can also have a watermark texture that matches the style of the original image.

[0144] Style can be sourced in two ways: First, the original image provider directly inputs style tags, such as "retro," "minimalist," and "cartoon." Technicians pre-configure the texture types and addition rules corresponding to each style tag, and the device adds textures according to the rules after retrieving the corresponding relationships. Second, a pre-trained style classification model extracts the style from the original image and automatically generates style tags. The style classification model can be built based on convolutional neural networks, such as Visual Geometry Group Network (VGGNet), Residual Network (ResNet), and Inception Network (Inception). After training and convergence using sample images and corresponding sample tags, automatic style recognition can be achieved.

[0145] In practical applications, a watermark base can be generated solely based on image brightness, or a base can be generated solely by adding textures based on a fixed brightness value. Alternatively, a style texture can be overlaid on a brightness-adapted base. The combination method can be flexibly selected according to the needs of the scenario.

[0146] In some embodiments, the keyword image carried on the watermark base can be further optimized in terms of three dimensions: font color, font type, and font size.

[0147] Regarding font color, a contrasting or complementary color to the main color of the original image can be used to further enhance the visual distinction between the text and the base / background, thereby improving readability; the main color of the image can be determined by calculating the average pixel value of the entire image, and then matching the corresponding contrasting or complementary color.

[0148] In terms of font type, in addition to preset fonts, font styles that match the style can also be generated through the large language model; by inputting the target keywords and prompts for the specified font generation rules into the large language model, the corresponding style of keyword display effect can be obtained, which enhances the overall visual aesthetics of the screen.

[0149] Regarding font size, the maximum available font size is adopted to ensure that the information is prominent, provided that the keywords are fully displayed within the target area of ​​the target watermark base. To avoid obscuring existing text in the original image, a graphic layout algorithm can be used to delineate the largest rectangular area that does not intersect with the text area of ​​the original image from the watermark base as the target area, and then the maximum available font size of the keywords is determined based on the size of the target area.

[0150] In practice, keyword images can be generated first, and then the font color can be adjusted to a contrasting color and scaled to fit the maximum size of the target area; alternatively, the font, color, and font size requirements can be directly written into the prompt words, and the large language model can directly generate keyword images that meet the requirements.

[0151] If the original image also contains core pattern areas such as products or faces, the rules for determining the target area can be further expanded. The largest rectangle that does not intersect with the text area or the pattern area can be defined as the target area, thereby reducing the probability of occlusion of the core content of the original image.

[0152] In some embodiments, see Figure 4 The image generation method includes the following steps: S401: Input processing module receives and processes the image; S402: Text extraction and semantic analysis module identifies whether there is text in the original image; if so, proceed to S403; S403: OCR (Optical Character Recognition) extracts all text, and analyzes and filters core marketing keywords; S404: Visual analysis; S405: Image tone / brightness analysis; S406: Automatically generate watermark base; S407: Text marketing and intelligent synthesis module determines whether there is a watermark base; if so, proceed to S408; S408: Keywords are rendered onto the watermark base, and the watermark is superimposed onto the original image in an intelligent blending mode; S409: Output image.

[0153] This invention, through an end-to-end automated intelligent processing pipeline, seamlessly integrates OCR, natural language processing, image analysis, graphics rendering, and compositing technologies to form a complete automated processing flow. It can perform batch and real-time intelligent optimization of massive amounts of raw image materials, greatly improving operational efficiency and reducing manual design and review costs.

[0154] In step S401, the input processing module receives the original image and performs preprocessing operations on it, including size standardization and image noise reduction. Size standardization adjusts the input original image to a preset standardized size, providing a unified size reference for subsequent processing by various modules.

[0155] In step S402, the text extraction and semantic analysis module identifies whether there is text in the original image; if there is text, step S403 is executed; if there is no text, semantic analysis is performed on the content of the original image to extract keywords.

[0156] For step S403, all text in the original image is extracted by optical character recognition (OCR) to obtain the aforementioned original text; then, a pre-trained keyword extraction model is used to extract each candidate keyword in the original text, and the predicted click probability corresponding to each keyword is obtained simultaneously; finally, from all candidate keywords, at least one keyword with the highest predicted click probability is selected as the core marketing keyword, i.e., the aforementioned target keyword.

[0157] In step S404, visual analysis processing is performed on the original image: the image generation device extracts the style label corresponding to the original image based on a pre-trained style classification model; then, from the preset correspondence between style label and watermark texture, it searches for the watermark texture that matches the style label, providing a basis for the texture adaptation of the subsequent watermark base.

[0158] For step S405, hue and brightness analysis processing is performed on the original image: the image generation device determines the main color tone and image brightness of the original image, where the main color tone is the dominant core color system in the original image, and the image brightness is the overall average brightness of the original image. Together, they provide a quantitative benchmark for the color and brightness configuration of the watermark base.

[0159] For step S406, the watermark base is automatically generated: the image generation device first generates a basic watermark base with corresponding brightness based on the image brightness of the original image; then the watermark texture matched by the visual analysis step is added to the basic base, and finally the target watermark base that matches the style and brightness of the original image is obtained.

[0160] In step S407, the copywriting and intelligent synthesis module determines whether a watermark base exists; if it exists, proceed to step S408; if it does not exist, the image generation device can directly add the target keywords to the original image without considering visual analysis and then perform fusion using fusion parameters.

[0161] For step S408, the copywriting and intelligent synthesis module performs watermark synthesis and image fusion processing: first, the keyword rendering to the watermark base is completed, that is, the selected target keywords are added to the target watermark base to obtain the complete target watermark; then, the intelligent blending mode of watermark overlay to the original image is executed, that is, the target watermark is added to the original image. During the overlay process, the edge of the target watermark can be feathered to weaken the harshness of the overlay boundary and make the fusion effect of the watermark and the original image more natural.

[0162] For step S409, the final processed image is output, namely the target image carrying the target keywords.

[0163] Based on the same inventive concept as the image generation method described above, embodiments of the present invention also provide an image generation apparatus. See [link to previous document]. Figure 5 The image generation apparatus includes:

[0164] The first extraction module 510 is used to identify the usage intent of the original image based on the semantics of the text in the original image, and extract target keywords that characterize the usage intent;

[0165] The second extraction module 520 is used to extract the visual features of the original image;

[0166] The determining module 530 is used to determine fusion parameters based on the visual features; wherein the fusion parameters are used to adapt to the visual style of the original image and to highlight the keywords in the original image;

[0167] The fusion module 540 is used to fuse the target keyword into the original image based on the fusion parameters.

[0168] Optionally, the first extraction module 510 includes:

[0169] The extraction unit is used to extract text from the original image to obtain the original text.

[0170] The acquisition unit is used to perform semantic analysis based on the original text, identify the usage intent of the original image, and extract candidate keywords that represent the usage intent; and obtain the predicted click probability corresponding to each candidate keyword based on the predicted click probability of the image displaying each candidate keyword.

[0171] The determining unit is used to determine at least one candidate keyword with the highest predicted click probability from each candidate keyword, as the target keyword representing the intended use.

[0172] Optionally, the acquisition unit is specifically used to, if there exists a preset keyword with a similarity greater than a preset threshold to each candidate keyword, use the preset click probability corresponding to the preset keyword as the predicted click probability corresponding to the candidate keyword; wherein, the preset click probability corresponding to each preset keyword represents the click frequency of the image displaying the preset keyword; if there is no preset keyword, the unit acquires the candidate click probability corresponding to the candidate keyword output by the prediction model as the predicted click probability of the candidate keyword; wherein, the prediction model is trained based on sample text, sample keywords in the sample text, and the actual click probability corresponding to each sample keyword.

[0173] Optionally, the fusion module 540 includes:

[0174] A generation unit is used to generate a watermark base based on a first parameter in the fusion parameters that describes the overall style of the original image.

[0175] An adjustment unit is used to adjust the target keyword according to a second parameter in the fusion parameters that describes the visual morphology of the keyword in the image;

[0176] The rendering unit is used to render the adjusted target keywords on the watermark base to obtain a fused watermark carrying the target keywords.

[0177] A fusion unit is used to fuse the fusion watermark in the original image based on a third parameter in the fusion parameters that describes the location of the keyword.

[0178] Optionally, the first parameter includes: the main color tone of the original image, the overall brightness information of the original image, and the style type of the original image;

[0179] The generation unit is specifically used to determine the color and transparency of the watermark base based on the main color tone and overall brightness information contained in the first parameter of the fusion parameters; generate the watermark base to be processed according to the determined color and transparency; determine the texture corresponding to the style type in the fusion parameters; and add the determined texture to the watermark base to be processed to obtain the desired watermark base.

[0180] Optionally, the third parameter is obtained by: identifying a preset protected area in the original image; determining the keyword location in the region of the original image other than the preset protected area, and obtaining a third parameter describing the keyword location.

[0181] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0182] Memory 603 is used to store computer programs;

[0183] When the processor 601 executes the program stored in the memory 603, it implements the steps of any of the image generation methods described in the foregoing embodiments.

[0184] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0185] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0186] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0187] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0188] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the image generation methods described in the above embodiments.

[0189] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image generation methods described in the above embodiments.

[0190] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0191] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0192] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0193] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An image generation method characterized by, The method includes: Based on the semantics of the text in the original image, the intended use of the original image is identified, and target keywords representing the intended use are extracted; Extract the visual features of the original image; Based on the visual features, fusion parameters are determined; wherein, the fusion parameters are used to adapt to the visual style of the original image and to highlight the keywords in the original image; Based on the fusion parameters, the target keywords are fused into the original image.

2. The method of claim 1, wherein, The process of identifying the usage intent of the original image based on the semantics of the text in the original image and extracting target keywords characterizing the usage intent includes: Extract the text from the original image to obtain the original text; Semantic analysis is performed on the original text to identify the intended use of the original image and extract candidate keywords that represent the intended use; and based on the predicted click probability of the image displaying each candidate keyword, the predicted click probability corresponding to that candidate keyword is obtained. From all the candidate keywords, at least one candidate keyword with the highest predicted click probability is selected as the target keyword representing the intended use.

3. The method according to claim 2, characterized in that, The step of obtaining the predicted click probability corresponding to each candidate keyword based on the predicted click probability of the image displaying each candidate keyword includes: If there is a preset keyword whose similarity to each candidate keyword is greater than a preset threshold, then the preset click probability corresponding to the preset keyword is used as the predicted click probability corresponding to that candidate keyword; wherein, the preset click probability corresponding to each preset keyword represents the click frequency of the image displaying that preset keyword; If the preset keyword does not exist, the candidate click probability corresponding to the candidate keyword output by the prediction model is obtained as the predicted click probability of the candidate keyword; wherein, the prediction model is trained based on sample text, sample keywords in the sample text, and the actual click probability corresponding to each sample keyword.

4. The method according to claim 1, characterized in that, The process of fusing the target keywords into the original image based on the fusion parameters includes: A watermark base is generated based on the first parameter in the fusion parameters that describes the overall style of the original image. The target keywords are adjusted according to the second parameter in the fusion parameters that describes the visual morphology of the keywords in the image; The adjusted target keywords are rendered on the watermark base to obtain a fused watermark carrying the target keywords; The fusion watermark is fused into the original image based on the third parameter in the fusion parameters that describes the location of the keywords.

5. The method according to claim 4, characterized in that, The first parameter includes: the main color tone of the original image, the overall brightness information of the original image, and the style type of the original image; The step of generating a watermark base based on the first parameter describing the overall style of the original image in the fusion parameters includes: Based on the primary color tone and overall brightness information contained in the first parameter of the fusion parameters, the color and transparency of the watermark base are determined. Generate the watermark base to be processed according to the determined color and transparency; Determine the texture corresponding to the style type in the fusion parameters; A defined texture is added to the watermark base to obtain the desired watermark base.

6. The method according to claim 4, characterized in that, The third parameter is obtained in the following manner: Identify a preset protected area in the original image; In the original image, outside the preset protected area, the location of the keyword is determined, and a third parameter describing the location of the keyword is obtained.

7. An image generation apparatus, characterized in that, The device includes: The first extraction module is used to identify the usage intent of the original image based on the semantics of the text in the original image, and extract target keywords that characterize the usage intent; The second extraction module is used to extract the visual features of the original image; A determining module is used to determine fusion parameters based on the visual features; wherein the fusion parameters are used to adapt to the visual style of the original image and to highlight the keywords in the original image; A fusion module is used to fuse the target keywords into the original image based on the fusion parameters.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a computer, the computer causes the computer to perform the method according to any one of claims 1-6.

Images