Poster background image selection, model training, poster generation method and related device

Abstract

The application discloses a poster background image selection, model training, poster generation method and related device. Through the application of the technical solution, a plurality of weakly related image text pairs can be used to train a preset visual text model, and a visual text model obtained through the training is used to automatically select a poster background image weakly related to the text information of interest to a user. Then, a final poster image is generated based on the automatically selected poster background image. Thus, the problem that a large number of high-quality poster requirements cannot be met by relying only on manual design to generate posters in the related art is avoided.

Description

Technical Field

[0001] This application relates to image data processing technology, and in particular to a method and apparatus for selecting poster background images, training models, generating posters. Background Technology

[0002] Posters, as a medium that combines artistry and functionality, are widely used in many commercial and non-commercial scenarios for advertising and disseminating information. For example, e-commerce platforms use attractive posters to promote their products.

[0003] In related technologies, existing poster generation methods typically involve professional graphic designers processing and arranging various product images and information. The specific process includes manually selecting the poster's background image, manually laying out the text, and manually determining the text style.

[0004] However, such a time-consuming and subjective process of manually designing and generating posters cannot meet the rapidly growing demand for high-quality posters in real-world applications, thus reducing the efficiency of information dissemination. Therefore, designing a technical solution that can automatically implement the above poster generation process using a pre-trained model has become a problem that needs to be solved. Summary of the Invention

[0005] This application provides a method and related apparatus for selecting poster background images, training models, and generating posters. This addresses the problem in related technologies where posters can only be generated manually, leading to reduced information dissemination efficiency.

[0006] According to one aspect of the embodiments of this application, a poster generation method is provided, comprising:

[0007] Obtain the poster description text, which contains text information to be added to the poster;

[0008] Based on a pre-trained visual text model, candidate background images that are weakly correlated with the text information are selected from a pre-acquired set of candidate background images as poster background images.

[0009] Optionally, in another embodiment based on the method described above in this application, the pre-trained visual text model includes: BriVL containing a pre-trained image encoder and a pre-trained text encoder;

[0010] The pre-trained visual text model selects candidate background images that are weakly correlated with the text information from a pre-acquired set of candidate background images as poster background images, including:

[0011] Based on the pre-trained text encoder, the pre-trained text encoder is used to extract features from the text information to obtain text features; and,

[0012] Based on the pre-trained image encoder, feature extraction is performed on each of the candidate background images to obtain the image features corresponding to each candidate background image;

[0013] Calculate the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and select the candidate background image with the highest weak correlation feature similarity to the text features as the poster background image.

[0014] Optionally, in another embodiment based on the method described above in this application, the pre-trained text encoder includes: the RoBERTa-Large encoder in the Chinese pre-trained model;

[0015] The pre-trained text encoder, used to extract features from the text information to obtain text features, includes:

[0016] The text information is input into the RoBERTa-Large encoder in the Chinese pre-trained model, so that the encoder can extract features from the text information and output the corresponding text features.

[0017] Optionally, in another embodiment based on the method described above in this application, the pre-trained image encoder includes: pre-trained Faster R-CNN and EfficientNet;

[0018] The pre-trained image encoder extracts features from each candidate background image to obtain image features corresponding to each candidate background image, including:

[0019] Each of the candidate background images is input into the pre-trained image encoder so that the pre-trained Faster R-CNN performs visual object detection processing on each of the candidate background images. Then, based on the EfficientNet, feature extraction is performed on each of the candidate background images after the visual object detection processing to obtain the image features corresponding to each of the candidate background images.

[0020] Optionally, in another embodiment based on the method described above in this application, the step of calculating the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and selecting the candidate background image with the highest weak correlation feature similarity to the text features to be compared as the poster background image, includes:

[0021] The InfoNCE loss function, constructed based on a pre-defined weakly supervised learning method and a contrastive learning method CPC, determines the weakly correlated feature similarity between the text features and the image features corresponding to each of the candidate background images.

[0022] According to one aspect of the embodiments of this application, a visual text model training method for selecting a poster background image is provided, comprising:

[0023] Multiple weakly correlated image-text pairs are obtained, wherein the weakly correlated image-text pairs are used to represent a set of weakly correlated candidate background images and historical text information;

[0024] Based on multiple weakly correlated image-text pairs, a preset visual text model is pre-trained and trained to obtain a visual text model for selecting poster background images that are weakly correlated with text information.

[0025] Optionally, in another embodiment based on the method described above in this application, the pre-trained visual text model includes: BriVL containing a pre-trained image encoder and a pre-trained text encoder;

[0026] The pre-trained text encoder includes: the RoBERTa-Large encoder in the Chinese pre-trained model; the pre-trained image encoder includes: pre-trained Faster R-CNN and EfficientNet.

[0027] The RoBERTa-Large encoder is used to extract features from each of the historical text information and output the corresponding text features.

[0028] The pre-trained Faster R-CNN is used to perform visual object detection processing on each of the candidate background images, and the EfficientNet is used to extract features from each of the candidate background images after the visual object detection processing to obtain the image features corresponding to each of the candidate background images.

[0029] The BriVL uses the InfoNCE loss function, constructed based on a preset weakly supervised learning method and the contrastive learning method CPC, to determine the weakly correlated feature similarity between the text features and the image features corresponding to each of the candidate background images.

[0030] According to one aspect of the embodiments of this application, a poster generation method is provided, comprising:

[0031] In the poster background image obtained based on the poster background image selection method described above, the text layout area corresponding to the text information is determined;

[0032] The text information is filled into the text layout area to generate the target poster corresponding to the text information.

[0033] Optionally, in another embodiment based on the method described above in this application, determining the text layout area in the poster background image includes:

[0034] The initial text layout area is determined by preliminary layout prediction of the poster background image using a first-level cascaded autoencoder; and...

[0035] Based on the text length and text attributes represented by the text information, the initial text layout area is refined by a second-level cascaded autoencoder to obtain the text layout area.

[0036] Optionally, in another embodiment based on the method described above in this application, filling the text information into the text layout area to generate a poster corresponding to the text information includes:

[0037] Extract the text features of the text information and detect the background color of the text layout area;

[0038] Based on the background color of the text layout area, a target text style that matches the text features and the background color is selected from a preset text style database;

[0039] The text information is converted according to the target text style and then filled into the text layout area to obtain the target poster corresponding to the text information.

[0040] According to another aspect of the embodiments of this application, a poster background image selection device is provided, characterized in that it includes:

[0041] The acquisition module is configured to acquire poster description text, which contains text information for adding to the poster;

[0042] The selection module is configured to select candidate background images that are weakly correlated with the text information from a pre-trained visual text model as poster background images.

[0043] According to another aspect of the embodiments of this application, an electronic device is provided, comprising:

[0044] Memory, used to store executable instructions; and

[0045] A display is configured to work with the memory to execute the executable instructions to perform any of the poster generation methods described above.

[0046] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided for storing computer-readable instructions, which, when executed, perform the operation of any of the poster generation methods described above.

[0047] In this application, user-generated poster description text containing text information to be added to the poster can be obtained. A set of candidate background images can be acquired, and based on a visual text model, candidate background images that are weakly correlated with the text information can be selected as the poster background images. The visual text model is trained from multiple sample images labeled with weakly correlated text pairs. After determining the text layout area in the poster background image, the text information is filled into the text layout area to obtain the target poster image. By applying the technical solution of this application, multiple weakly correlated image-text pairs can be used to train a preset visual text model, and the trained visual text model can be used to automatically select poster background images that are weakly correlated with the text information of interest to the user. Subsequently, the final poster image is generated based on the automatically selected poster background image. This avoids the problem in related technologies where relying solely on manual poster design and generation fails to meet the large demand for high-quality posters.

[0048] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] The accompanying drawings, which form part of this specification, illustrate embodiments of this application and, together with the description, serve to explain the principles of this application.

[0050] This application can be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0051] Figure 1 This is a schematic diagram illustrating a method for selecting a poster background image as proposed in this application;

[0052] Figure 2 This is a schematic diagram illustrating the process of selecting a poster background image according to the present application;

[0053] Figure 3 This is a schematic diagram of a visual text model training method for selecting poster background images proposed in this application;

[0054] Figure 4 This is a schematic diagram illustrating a poster generation method proposed in this application;

[0055] Figure 5 A reference figure is provided for an example of selecting a poster background image according to this application;

[0056] Figure 6A schematic diagram comparing the text layout method proposed in this application with the text layout methods of existing technologies;

[0057] Figure 7 This is a flowchart illustrating a poster text layout proposed in this application;

[0058] Figure 8 This is a schematic diagram illustrating the overall process of a poster generation method proposed in this application.

[0059] Figure 9 A schematic diagram of the structure of an electronic device for selecting a poster background image according to this application;

[0060] Figure 10 This is a schematic diagram of the structure of an electronic device for selecting a poster background image according to the present application. Detailed Implementation

[0061] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0062] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0063] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this application or its application or use.

[0064] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0065] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0066] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0067] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0068] The following is combined Figures 1-8 This application describes a method for generating a poster according to exemplary embodiments thereof. It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario.

[0069] This application also proposes a method and related apparatus for selecting poster background images, training models, and generating posters.

[0070] Figure 1 A schematic flowchart illustrating a poster generation method according to an embodiment of this application is shown. Figure 1 As shown, the method includes:

[0071] S101, Obtain the poster description text, which contains text information to be added to the poster.

[0072] In related technologies, posters, as a medium that combines artistry and functionality, have been widely used in many commercial and non-commercial scenarios for advertising and disseminating information. For example, e-commerce platforms use attractive posters to promote their products. Websites for social events such as conferences are often decorated with beautifully designed and content-rich posters. These high-quality posters are generated by embedding stylized text into appropriate background images, a process that requires significant manual editing work and art design experience that is difficult to quantify. However, such a time-consuming and subjective manual design process cannot meet the large and rapidly growing demand for high-quality posters in real-world applications, thus reducing the efficiency of information dissemination and resulting in poor advertising effectiveness.

[0073] In one approach, the poster generation process involves at least three steps, including:

[0074] Step 1: Select the background image for the poster;

[0075] Step 2: Arrange the text in the poster background image;

[0076] Step 3: Determine the style of the text on the poster and fill it into the background image to generate the final poster image.

[0077] Based on the above process, specifically for step 1, in order to achieve automatic search for the background image of the poster, the existing classic retrieval methods generally search for suitable images by matching the poster content text with the background image annotation text.

[0078] However, methods in highly relevant techniques typically only consider single-modal features, thus often failing to bridge the semantic gap between visual and textual modalities and producing biased search results. In other words, the selected background image does not match the user's current area of ​​interest.

[0079] Furthermore, regarding step 2, in order to predict the text layout, conventional rule-based methods typically select the text layout from a limited number of predefined layout templates, which lacks flexibility. This results in the background image content being ignored when arranging the text, potentially leading to unsatisfactory poster generation results.

[0080] Furthermore, in order to solve the problem of mismatch between the selected poster background image and the user's current interest, this application proposes a technical solution for selecting poster background images based on a pre-trained visual text model of Text2Poster.

[0081] In summary, this application can first obtain a large-scale pre-trained visual text model, and select a candidate background image that matches the poster description text from among many candidate background images based on the poster description text given by the user as the poster background image. Then, the text is iteratively arranged on the image through a cascaded autoencoder, and finally the text is stylized by a matching-based method to synthesize the target poster.

[0082] In one approach, this application can also optimize each module of the framework through weakly supervised and self-supervised learning strategies, thereby reducing the dependence on label data. This achieves the goal of making the technical solution based on the data-driven framework Text2Poster perform better in terms of generating poster quality.

[0083] Specifically, this application can first obtain user-generated poster description text containing text information to be added to the poster. It is understood that this poster description text is a description of the poster information the user wants to generate. It may include multiple parameters, such as title information, scene information, implementation method information, character information, etc.

[0084] S102, based on a pre-trained visual text model, selects candidate background images that are weakly correlated with the text information from a pre-acquired set of candidate background images as poster background images.

[0085] In one approach, to improve the quality of the poster, this embodiment of the application may, during the poster generation process, search for images that have a weak correlation with the text information (i.e., the text information contained in the poster description text that is used to add to the poster) when performing background image retrieval.

[0086] Specifically, weak correlation matching refers to metaphorical matching. For example, when searching for background images based on the phrase "Bob and Alice's wedding," this application tends to find images with more metaphorical meaning rather than images of a specific wedding scene (i.e., non-strongly correlated), such as an image of a white church under a blue sky that metaphorically represents love.

[0087] Furthermore, to achieve the goal of selecting candidate background images that are weakly correlated with text information from the candidate background image set as the poster background image, this application can utilize BriVL, one of the pre-trained visual text models, to select the background image from the candidate images based on text information.

[0088] Specifically, for example Figure 2 As shown, BriVL is the visual text model proposed in this application. It can be seen that it consists of an image encoder and a text encoder, denoted as f respectively. I (i.e., image encoder) and f T (i.e., text encoder).

[0089] Wherein, the image encoder f I First, a pre-trained Faster R-CNN model is used to detect visual objects. Then, the EfficientNet model is applied as its main visual framework to extract image features corresponding to each background candidate image.

[0090] Additionally, the text encoder f T The RoBERTa-Large encoder from the Chinese pre-trained model can then be used as its main text framework. Based on the output of the aforementioned main framework model, BriVL stacks multiple Transformers to derive the text features corresponding to the text information.

[0091] It should be noted that the visual text model BriVL in this application needs to be pre-trained on multiple (e.g., 30 million) weakly correlated "sample image-text pairs" collected from the network, thus satisfying the weakly correlated artistry of the poster generation results proposed in this application.

[0092] Understandably, this application employs a weakly supervised learning strategy and the InfoNCE loss function to align text features with image features. The loss function constructed using the contrastive learning method CPC (Contrastive Predictive Coding) is InfoNCE, where NCE stands for Noise Contrastive Estimation.

[0093] In one approach, this application can collect multiple high-quality images from image resource websites or other sources as a candidate background image set. This allows for the subsequent selection of poster background images based on the BriVL visual text model mentioned in this application.

[0094] Specifically, since this application extracts the image features corresponding to each candidate background image and the text features corresponding to the text information, it can calculate the weak correlation feature similarity between the text features and the image features corresponding to each candidate background image, and then use the candidate background image with the highest weak correlation feature similarity to the text features as the poster background image.

[0095] In this application, user-generated poster description text containing text information to be added to the poster can be obtained. A set of candidate background images can be acquired, and based on a visual text model, candidate background images that are weakly correlated with the text information can be selected as the poster background images. The visual text model is trained from multiple sample images labeled with weakly correlated text pairs. After determining the text layout area in the poster background image, the text information is filled into the text layout area to obtain the target poster image. By applying the technical solution of this application, multiple weakly correlated image-text pairs can be used to train a preset visual text model, and the trained visual text model can be used to automatically select poster background images that are weakly correlated with the text information of interest to the user. Subsequently, the final poster image is generated based on the automatically selected poster background image. This avoids the problem in related technologies where relying solely on manual poster design and generation fails to meet the large demand for high-quality posters.

[0096] Optionally, in another embodiment based on the method described above in this application, the pre-trained visual text model includes: BriVL containing a pre-trained image encoder and a pre-trained text encoder;

[0097] Based on a pre-trained visual text model, candidate background images that are weakly correlated with the text information are selected from a pre-acquired set of candidate background images as poster background images, including:

[0098] Based on a pre-trained text encoder, the pre-trained text encoder extracts features from text information to obtain text features; and,

[0099] The pre-trained image encoder extracts features from each candidate background image to obtain the image features corresponding to each candidate background image.

[0100] Calculate the weak correlation feature similarity between the text features and the image features corresponding to each candidate background image, and select the candidate background image with the highest weak correlation feature similarity to the text features as the poster background image.

[0101] Still based on the above Figure 2 For example, BriVL is the visual text model proposed in this application. It can be seen that it consists of an image encoder and a text encoder, denoted as f respectively. I (i.e., image encoder) and f T (i.e., text encoder).

[0102] Wherein, the image encoder f I First, a pre-trained Faster R-CNN model is used to detect visual objects. Then, the EfficientNet model is applied as its main visual framework to extract image features corresponding to each background candidate image.

[0103] Additionally, the text encoder f T The RoBERTa-Large encoder from the Chinese pre-trained model can then be used as its main text framework. Based on the output of the aforementioned main framework model, BriVL stacks multiple Transformers to derive the text features corresponding to the text information.

[0104] Specifically, in this embodiment of the application, after obtaining the image features corresponding to each candidate background image and the text features corresponding to the text information contained in the poster description text, the multiple features can be converted into corresponding encoded vector values ​​for representation.

[0105] For example, when the encoding vector r of the text features is obtained T' and the encoded vector of each candidate background image Then you can use r T' =f T (U i T i ), To express.

[0106] In one embodiment of this application, r can be calculated. T' With each The cosine similarity between the images is used to select the candidate background image with the highest similarity as the poster background image I.

[0107] The formula for calculating the cosine similarity between feature vectors can be:

[0108]

[0109] Optionally, in another embodiment based on the method described above in this application, the pre-trained text encoder includes: a RoBERTa-Large encoder in a Chinese pre-trained model;

[0110] Based on a pre-trained text encoder, the text encoder is used to extract features from text information, resulting in text features, including:

[0111] The text information is input into the RoBERTa-Large encoder in the Chinese pre-trained model so that the encoder can extract features from the text information and output the corresponding text features.

[0112] Optionally, in another embodiment based on the method described above in this application, the pre-trained image encoder includes: pre-trained Faster R-CNN and EfficientNet;

[0113] The pre-trained image encoder extracts features from each candidate background image to obtain the image features corresponding to each candidate background image, including:

[0114] Each candidate background image is input into a pre-trained image encoder so that the pre-trained Faster R-CNN performs visual object detection on each candidate background image. Then, based on EfficientNet, features are extracted from each candidate background image after visual object detection to obtain the image features corresponding to each candidate background image.

[0115] Optionally, in another embodiment based on the method described above in this application, the step of calculating the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and selecting the candidate background image with the highest weak correlation feature similarity to the text features to be compared as the poster background image, includes:

[0116] The InfoNCE loss function, constructed based on the pre-defined weakly supervised learning method and the contrastive learning method CPC, determines the weakly correlated feature similarity between text features and the image features corresponding to each candidate background image.

[0117] It should be noted that the visual text model BriVL in this application needs to be pre-trained on multiple (e.g., 30 million) weakly correlated "sample image-text pairs" collected from the network, thus satisfying the weakly correlated artistry of the poster generation results proposed in this application.

[0118] In one approach, during the weak correlation matching process using image features and text features of candidate background images, the candidate background image with the highest similarity to the weakly correlated features of the text features to be compared can be used as the poster background image. Specifically, this application requires applying a weakly supervised learning strategy and the InfoNCE loss function to perform feature alignment between the text features and the image features of each candidate background image. After feature alignment, weak correlation matching is then performed between the two.

[0119] For the InfoNCE loss function, this application can use the loss function constructed by the contrastive learning method CPC (Contrastive Predictive Coding) as the InfoNCE, where NCE refers to Noise Contrastive Estimation.

[0120] By applying the technical solution of this application, multiple weakly correlated image-text pairs can be trained on a preset visual text model. The trained visual text model can then be used to automatically select poster background images that are weakly correlated with text information of interest to the user. The final poster image is then generated based on this automatically selected background image. This avoids the problem in related technologies where relying solely on manual poster design fails to meet the large demand for high-quality posters.

[0121] Figure 3 A schematic flowchart illustrating a visual text model training method for selecting a poster background image according to an embodiment of this application is shown. Figure 3 As shown, the method includes:

[0122] S201, Obtain multiple weakly correlated image-text pairs, wherein the weakly correlated image-text pairs are used to represent a set of weakly correlated candidate background images and historical text information.

[0123] S202, based on multiple weakly correlated image-text pairs, a pre-trained and trained visual text model is obtained to obtain a visual text model for selecting poster background images that are weakly correlated with text information.

[0124] In one approach, during the training of the visual text model, this application can use multiple pre-acquired weakly correlated image-text pairs as training samples. It should be noted that these weakly correlated image-text pairs represent a set of weakly correlated candidate background images and historical text information. That is, each candidate background image is labeled with a set of text descriptions that are weakly correlated with it.

[0125] For example, a weakly correlated image-text pair may include: a candidate background image of blue sky and white clouds and the corresponding text information "wedding". The text information "wedding" is the historical text information weakly correlated with the candidate background image of blue sky and white clouds.

[0126] Alternatively, a weakly correlated image-text pair could be a candidate background image of a school classroom and the corresponding labeled text "growth". Here, the text "growth" is the historical text information weakly correlated with the candidate background image of the school classroom. Understandably, by using these multiple weakly correlated image-text pairs as sample training data to pre-train and train the initial preset visual text model, a visual text model that satisfies the artistic requirement of a weak correlation between the text information and the poster background image can be obtained.

[0127] It should be noted that this application does not specify the number of weakly correlated image-text pairs, such as 10 million or 30 million, etc.

[0128] In one approach, when the present application is pre-training and training a preset visual text model based on multiple weakly correlated image-text pairs, it is determined that the final visual text model has been obtained when the preset training conditions are detected (e.g., reaching a certain number of training times, reaching a certain training time, training has converged, etc.).

[0129] In this application, multiple weakly correlated image-text pairs can be obtained, wherein the weakly correlated image-text pairs are used to represent a set of weakly correlated candidate background images and historical text information; based on the multiple weakly correlated image-text pairs, a preset visual text model is pre-trained and trained to obtain a visual text model for selecting poster background images that are weakly correlated with text information.

[0130] By applying the technical solution of this application, multiple weakly correlated image-text pairs can be trained on a preset visual text model. The trained visual text model can then be used to automatically select poster background images that are weakly correlated with text information of interest to the user. The final poster image is then generated based on this automatically selected background image. This avoids the problem in related technologies where relying solely on manual poster design fails to meet the large demand for high-quality posters.

[0131] Alternatively, in another embodiment based on the method described above in this application,

[0132] The pre-trained visual text model includes BriVL, which contains a pre-trained image encoder and a pre-trained text encoder.

[0133] The pre-trained text encoders include the RoBERTa-Large encoder in the Chinese pre-trained model, and the pre-trained image encoders include the pre-trained Faster R-CNN and EfficientNet.

[0134] The RoBERTa-Large encoder is used to extract features from each historical text and output the corresponding text features.

[0135] The pre-trained Faster R-CNN is used to perform visual object detection processing on each candidate background image, and EfficientNet is used to extract features from each candidate background image after visual object detection processing to obtain the image features corresponding to each candidate background image.

[0136] BriVL uses the InfoNCE loss function, built based on a pre-defined weakly supervised learning method and the contrastive learning method CPC, to determine the weakly correlated feature similarity between text features and the image features corresponding to each candidate background image.

[0137] In one approach, the visual text model in this application can be constructed based on BriVL, which consists of a pre-trained image encoder and a pre-trained text encoder.

[0138] The pre-trained image encoder first uses a pre-trained Faster R-CNN model to detect visual objects, and then applies the EfficientNet model as its main visual framework to extract image features corresponding to each background candidate image.

[0139] Alternatively, the pre-trained text encoder can use the RoBERTa-Large encoder from the Chinese pre-trained model as its main text framework. Based on the output of the aforementioned main framework model, BriVL stacks multiple Transformers to extract features from each historical text and output the corresponding text features.

[0140] It should be noted that this application can apply a weakly supervised learning strategy and the InfoNCE loss function to align the features of the text with the features of the image. The loss function constructed by the contrastive learning method CPC (Contrastive Predictive Coding) is InfoNCE, where NCE stands for Noise Contrastive Estimation.

[0141] Figure 4 A schematic flowchart illustrating a poster generation method according to an embodiment of this application is shown. Figure 4 As shown, the method includes:

[0142] S301, Determine the text layout area corresponding to the text information.

[0143] S302, fill the text information into the text layout area to generate the target poster corresponding to the text information.

[0144] In one approach, this application uses a pre-trained visual text model to select candidate background images that are weakly correlated with the text information from a pre-acquired set of candidate background images as poster background images. Then, the text information is filled into the text layout area of ​​the poster background image to obtain the final target poster. Specifically, this includes the following steps:

[0145] Step 1: Select a candidate background image as the poster background image.

[0146] Among them, BriVL, one of the pre-trained visual text models, is used to select background images from candidate images based on text information.

[0147] Step 2: Determine the text layout area.

[0148] Among them, the textual information T contained in the poster description text i Perform text layout prediction separately. Let it be denoted as...

[0149] Where, p i ∈R 2 , representing the i-th text information T i The normalized top-left corner coordinates of the poster background image I are then used to form the text information T. i The text layout prediction results.

[0150] Step 3: Text style determination: Determine the text features of each text information (such as text font and text color), and fill each text information into its corresponding text layout area according to the text layout prediction results to complete the poster generation.

[0151] Optional, Figure 5 This diagram illustrates the method proposed in this application for finding background images that have a weak correlation with text information (i.e., text information contained in the poster description text for use on the poster). It shows the background images selected for each piece of text information.

[0152] Optionally, in the poster generation method provided in this application, it is first necessary to implement a poster text layout method and, based on this poster text layout method, implement the text layout in the poster image to generate the corresponding target poster image. Specifically, the poster text layout method in this application may include the following steps:

[0153] Determine the smooth area in the poster background image corresponding to the poster description text, and use the first autoencoder to select a usable area in the smooth area;

[0154] The available area is sampled, and based on the corresponding sampling results and the text information to be added to the poster corresponding to the poster description text, a text box corresponding to the available area is generated to obtain the target text layout area in the poster background image for writing text information.

[0155] Optionally, determine the smooth area in the poster background image corresponding to the poster description text, including:

[0156] Multiple candidate boxes are selected in the poster background image; and a salient image corresponding to the poster background image is generated using a spectral residual algorithm, wherein there are overlapping candidate boxes among the candidate boxes.

[0157] Based on the salient mean of the salient image, the number of pixels in the candidate box, and a preset offset, a candidate value is determined for each candidate box.

[0158] Candidate boxes with candidate values ​​lower than a preset candidate threshold are selected as target candidate boxes. Based on the target candidate boxes, smooth regions in the poster background image are determined, wherein the positions of each target candidate box in the poster background image do not overlap.

[0159] Optionally, candidate boxes with candidate values ​​lower than a preset candidate threshold are selected as target candidate boxes, and based on the target candidate boxes, smooth regions in the poster background image are determined, including:

[0160] Based on the nonmaximum suppression method, candidate boxes with candidate values ​​lower than a preset candidate threshold are selected as target candidate boxes;

[0161] Determine the candidate box regions in the poster background image where all target candidate boxes are located;

[0162] The candidate box region in the poster background image is converted into a binary image, and the converted binary image region is used as the smooth region in the poster background image.

[0163] Optionally, a usable area is selected in the smooth region using a first autoencoder, including:

[0164] Each target candidate box corresponding to the smooth region is used as the encoder input of the first autoencoder to obtain the encoder output, wherein the encoder is constructed by stacked CNNs;

[0165] After concatenating the encoder output with the location embedding map, it is used as the input to the decoder of the first autoencoder to obtain the available area in the smooth region as the initial text layout result, where the decoder is constructed from stacked Transposed-CNNs.

[0166] Optionally, after generating the text boxes corresponding to the available areas, the following may also be included:

[0167] Using a second autoencoder, the text boxes corresponding to each available area are refined and laid out in an autoregressive manner to obtain the target text boxes corresponding to each available area, thus forming the target text layout area in the poster background image for writing text information.

[0168] Optionally, both the first and second autoencoders are layout predictors with a cascaded autoencoder architecture.

[0169] Optionally, multiple sample images are acquired, each of which contains a text region labeled with a corresponding text description field;

[0170] Extract the sample background image from each sample image, and use a smooth region detector to determine the sample smooth image region in the sample background image;

[0171] The text description field, text region, sample background image, and sample smoothed image region are merged into the encoder training dataset;

[0172] The first autoencoder and the second autoencoder are trained independently using the encoder training dataset, so that the first autoencoder is used to predict the probability distribution of the text layout and the second autoencoder is used to refine the layout box.

[0173] Optionally, the second autoencoder further includes: trained using a self-supervised learning strategy, wherein the encoder end of the second autoencoder is constructed from stacked CNNs, and the decoder end of the second autoencoder is constructed from a two-layer bidirectional LSTM.

[0174] Optional, Figure 6This diagram illustrates a comparison between the poster text layout method proposed in this application and other text layout methods in the prior art. It shows the target poster image corresponding to each poster text layout.

[0175] In this application, user-generated poster description text containing text information to be added to the poster can be obtained. A set of candidate background images can be acquired, and based on a visual text model, candidate background images that are weakly correlated with the text information can be selected as the poster background images. The visual text model is trained from multiple sample images labeled with weakly correlated text pairs. After determining the text layout area in the poster background image, the text information is filled into the text layout area to obtain the target poster image. By applying the technical solution of this application, multiple weakly correlated image-text pairs can be used to train a preset visual text model, and the trained visual text model can be used to automatically select poster background images that are weakly correlated with the text information of interest to the user. Subsequently, the final poster image is generated based on the automatically selected poster background image. This avoids the problem in related technologies where relying solely on manual poster design and generation fails to meet the large demand for high-quality posters.

[0176] Optionally, in another embodiment based on the method described above in this application, determining the text layout area in the poster background image includes:

[0177] The initial text layout area is determined by preliminary layout prediction on the poster background image using a first-level autoencoder; and...

[0178] Based on the text length and text attributes represented by the text information, the initial text layout area is refined through a second-level cascaded autoencoder to obtain the text layout area.

[0179] In one approach, after obtaining the poster background image I, this application can predict the text layout region P of the text information using two cascaded autoencoders (i.e., a first cascaded autoencoder and a second cascaded autoencoder), specifically including the following steps:

[0180] Step a: Identify the smooth image area in the poster background image.

[0181] Furthermore, this application first needs to generate several regions (candidate boxes) of different sizes in the background image I. These regions overlap, denoted as... Then, the spectral residual method is applied to generate the saliency map corresponding to the background image I, denoted as S.

[0182] For each candidate box, a candidate value v is assigned to each candidate box by calculating the average value of the saliency map S. i .Right now

[0183] Where S(p) refers to the significance of the text layout prediction result at point p, |A i |Is A i The number of pixels in the bounding box, λ is an offset that is sensitive to the size of the candidate bounding box.

[0184] In one preferred embodiment, this application selects candidate boxes with candidate values ​​lower than a preset candidate threshold as target candidate boxes. It then determines the candidate box region within the poster background image where the target candidate box is located, and converts this region into a binary image. Finally, the converted binary image region is used as the smoothed image region in the poster background image.

[0185] It should be noted that the preset candidate threshold can be adaptively set according to different images. That is, the threshold can be set according to the average value of each region of the current background image I. For example, the threshold can be adaptively set to 1.4×mean{}, and the non-maximum suppression method (NMS) can be applied to ensure that the regions finally selected from the previously generated regions do not overlap (for example, if 1000 candidate boxes are generated in the early stage, these boxes may overlap).

[0186] For example, such as Figure 7 As shown, the more central part of image (a) is the salient map S of the background image I, which can be displayed using a first color (e.g., blue) in practical applications. The more peripheral rectangular area in image (a) is the smooth area map A obtained through this step, which can be displayed using a second color (e.g., red) that is clearly different from the first color in practical applications.

[0187] Step b: Use a first autoencoder to determine at least one initial text layout region in the smooth image region.

[0188] In one approach, this application can use a first cascaded autoencoder g1 to predict the probability distribution used for text layout based on the smooth region A obtained in step a above, denoted as L. For each pixel p, L(p)∈[0,1] represents the probability that p belongs to a text box.

[0189] In this application, the first cascaded autoencoder g1 has an autoencoder architecture, where the encoder f1 is a stacked CNN and the decoder h1 is a stacked Transposed-CNN. The input to the decoder h1 is then constructed by concatenating the output of the encoder f1 with a learnable location embedding map (denoted as E).

[0190] For example Figure 3 The whitish portion in image (b) shows the probability distribution prediction result L of text layout in the poster background image I, where the brighter the color, the higher the layout probability.

[0191] Step c: Using a second autoencoder, iteratively refine the layout of at least one initial text layout region in an autoregressive manner to obtain the target text layout region.

[0192] The layout prediction probability distribution result L obtained in the above steps is the initial area selection for text layout. It needs to be refined to form clear text boxes for each available area.

[0193] That is, firstly, the layout is initialized based on L by sampling each element from L. The unnormalized coordinates are used as the top-left corner coordinates of the i-th text box, and the box is initialized. Its size is determined by the length and attributes of the corresponding text information. Finally, the position and size of each text box used to write each text information are determined in the background image I, which is the text layout prediction result.

[0194] In one specific implementation, this application can utilize a second-level cascaded autoencoder to refine the initial text layout region to obtain the final text layout region, including:

[0195] P (k+1) =g2(Concat(A,L),P (k) ), k = 0, ..., K-1.

[0196] Where k is the number of iterations, g2 is the second autoencoder, A is the smoothed image region, L is the probability distribution of the initial text layout region, and P is the pixel in the poster background image.

[0197] It should be noted that for the second-stage cascaded autoencoder, its encoder can be a stacked CNN, and its decoder can be a two-layer bidirectional LSTM.

[0198] In a preferred embodiment, both the first cascaded autoencoder for predicting the available area layout in step b and the second cascaded autoencoder for refining the text layout in step c can employ a layout predictor with a cascaded autocoding architecture, thereby achieving the goal of effectively mimicking the manual image editing process.

[0199] like Figure 8 The diagram shown is the overall flowchart of the poster generation method proposed in this application.

[0200] Optionally, in another embodiment based on the method described above, filling text information into a text layout area to generate a poster corresponding to the text information includes:

[0201] Extract text features from text information and detect the background color of text layout areas;

[0202] Based on the background color of the text layout area, select the target text style that matches the text features and background color from the preset text style database.

[0203] After converting the text information according to the target text style, it is filled into the text layout area to obtain the target poster corresponding to the text information.

[0204] Furthermore, this application can process each textual information T in the poster description text. i For each ∈T', its corresponding textual feature r = f needs to be extracted. T (T), and obtain the background color of the text features as c I =I(p). Where p i It is text information T i The corresponding text layout area.

[0205] Additionally, based on (r,c) T It can search for target text styles that match the text features and background color in a preset text style database F under cosine similarity, thereby determining each T. i Choose the color and font, and then fill them into the corresponding text layout area.

[0206] Specifically, the following are the implementation details of the two autoencoders g1 and g2 in this application, including:

[0207] A) Each convolutional layer used in g1 contains 16 convolutional kernels of size 9×9; the encoder of g1 finally outputs a 64-dimensional feature vector.

[0208] B) Each convolutional layer of the g2 encoder contains 64 convolutional kernels of size 5×5; the hidden layers of the g2's 2-layer bidirectional LSTM (decoder) are set to 200 dimensions.

[0209] C) When training the two autoencoders mentioned above, the training and validation sets can be divided. For example, the dataset D can be divided into 138,013 training poster images and 16,000 validation poster images. The size of each poster image can be adjusted to 300×400. The Adam algorithm can be used to optimize the autoencoder models g1 and g2, with a learning rate of 0.05 and a batch size of 512. The two autoencoders were trained for 4 (g1) and 48 (g2) hours respectively on four V100 GPUs.

[0210] Finally, to demonstrate the effectiveness of the Text2Poster framework proposed in this application, the following section also includes corresponding verification, which is explained in detail below:

[0211] 1) Validation for image retrieval:

[0212] First, the rationality of (i) the image retrieval method is verified. For ease of comparison, in addition to the BriVL method provided in this application, this application also refers to (a) using the search engine unsplash.com and (b) matching the input text with image tags in the image retrieval library of this application as control groups.

[0213] This application presents representative search results obtained using three different search methods. Compared to the other two methods, the images retrieved using the BriVL method provided in this application do indeed contain metaphors corresponding to the input text. For example, given the text "Campus Charity Sale," the search engine on unsplash.com tends to search for images with explicit concepts such as "sale" and "campus," while the BriVL method provided in this application retrieves images of growing trees, rainbows, and colored handprints, which are metaphorical but semantically appropriate to "Campus Charity Sale." Even for challenging abstract descriptions like "See the world together" and "Dreams never stop," the BriVL method provided in this application can still find suitable images.

[0214] In the subjective evaluation, given 50 text queries, this application retrieved the top 5 images for each query using different methods. Three volunteers were invited to rate the quality of the retrieved images from 0 (very poor) to 4 (very good). The average score and standard deviation of the Unsplash.com search engine were 2.17 ± 0.10, the average score and standard deviation of the tag-based matching method were 1.64 ± 0.16, and the average score and standard deviation of the BriVL method provided in this application were 2.38 ± 0.13, further demonstrating the superiority of the method presented in this application.

[0215] 2) Validation of text layout prediction:

[0216] This application quantitatively and qualitatively evaluates the layout predictor proposed in this application and compares it with the following control group:

[0217] A) The state-of-the-art learning-based method, LayoutGAN++;

[0218] B) State-of-the-art rule-based methods IUI and DeSal;

[0219] C) LUBAN, a business poster generator on https: / / luban.aliyun.com.

[0220] Furthermore, to demonstrate the usefulness of the iterative layout optimization strategy of this application, K is set to 1, 5 and 30 respectively for the layout predictor of this application.

[0221] This application collects 16,000 posters from huaban.com to construct a reference dataset and prepares three background image sets: Unsplash2K, Unsplash10K, and PSD1.6K. Unsplash2K and Unsplash10K contain 2,000 and 10,000 background images from unsplash.com, respectively. PSD1.6K contains 1,637 background images extracted from poster files in PSD format. For each image set, this application arranges the input text on the background images using various methods and generates posters. This application calculates the initial distance FID between the posters and the reference dataset according to the work in LayoutGAN++. The results in Table 1 show that the method of this application consistently outperforms the control group mentioned above, and its performance improves with increasing K, which verifies the rationality of the iterative refinement strategy of this application.

[0222] Table 1

[0223]

[0224] Table 1 shows the objective and subjective evaluations of various layout prediction methods. LUBAN in (C) only offers a paid service and cannot be used for large-scale quantitative evaluation.

[0225] In addition, this application manually selected 50 text sets, each containing a title and several subheadings or descriptions.

[0226] For each layout prediction method, this application first retrieves five background images for each set of text and generates 250 posters accordingly. Three volunteers are asked to rate the layout aesthetics of these generated posters from 0 (very poor) to 4 (very good). The mean and standard deviation of the scores for each method are shown in Table 1.

[0227] Optionally, in another embodiment of this application, such as Figure 9 As shown, this application also provides a poster background image selection device. This includes:

[0228] The acquisition module 401 is configured as an acquisition module to acquire poster description text, which contains text information for adding to the poster;

[0229] The selection module 402 is configured to select candidate background images that are weakly correlated with the text information from a pre-trained visual text model as poster background images.

[0230] In this application, user-generated poster description text containing text information to be added to the poster can be obtained. A set of candidate background images can be acquired, and based on a visual text model, candidate background images that are weakly correlated with the text information can be selected as the poster background images. The visual text model is trained from multiple sample images labeled with weakly correlated text pairs. After determining the text layout area in the poster background image, the text information is filled into the text layout area to obtain the target poster image. By applying the technical solution of this application, multiple weakly correlated image-text pairs can be used to train a preset visual text model, and the trained visual text model can be used to automatically select poster background images that are weakly correlated with the text information of interest to the user. Subsequently, the final poster image is generated based on the automatically selected poster background image. This avoids the problem in related technologies where relying solely on manual poster design and generation fails to meet the large demand for high-quality posters.

[0231] In another embodiment of this application, the steps configured to be executed by the acquisition module 401 include:

[0232] The pre-trained visual text model includes: BriVL containing a pre-trained image encoder and a pre-trained text encoder;

[0233] The pre-trained visual text model selects candidate background images that are weakly correlated with the text information from a pre-acquired set of candidate background images as poster background images, including:

[0234] Based on the pre-trained text encoder, the pre-trained text encoder is used to extract features from the text information to obtain text features; and,

[0235] Based on the pre-trained image encoder, feature extraction is performed on each of the candidate background images to obtain the image features corresponding to each candidate background image;

[0236] Calculate the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and select the candidate background image with the highest weak correlation feature similarity to the text features as the poster background image.

[0237] In another embodiment of this application, the selected module 402 is configured to perform the following steps:

[0238] The pre-trained text encoder, used to extract features from the text information to obtain text features, includes:

[0239] The text information is input into the RoBERTa-Large encoder in the Chinese pre-trained model, so that the encoder can extract features from the text information and output the corresponding text features.

[0240] In another embodiment of this application, the steps configured to be executed by the acquisition module 401 include:

[0241] The pre-trained image encoders include: pre-trained Faster R-CNN and EfficientNet;

[0242] The pre-trained image encoder extracts features from each candidate background image to obtain image features corresponding to each candidate background image, including:

[0243] Each of the candidate background images is input into the pre-trained image encoder so that the pre-trained Faster R-CNN performs visual object detection processing on each of the candidate background images. Then, based on the EfficientNet, feature extraction is performed on each of the candidate background images after the visual object detection processing to obtain the image features corresponding to each of the candidate background images.

[0244] In another embodiment of this application, the selected module 402 is configured to perform the following steps:

[0245] The InfoNCE loss function, constructed based on a pre-defined weakly supervised learning method and a contrastive learning method CPC, determines the weakly correlated feature similarity between the text features and the image features corresponding to each of the candidate background images.

[0246] Figure 10 This is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0247] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions. These instructions can be executed by a processor of an electronic device to complete the poster generation method described above. The method includes: acquiring poster description text, which contains text information for adding to the poster; and selecting candidate background images that weakly match the text information from a pre-trained visual text model set as poster background images. Optionally, the instructions can also be executed by a processor of an electronic device to complete other steps involved in the exemplary embodiment described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0248] In an exemplary embodiment, an application / computer program product is also provided, including one or more instructions that can be executed by a processor of an electronic device to complete the poster generation method described above. The method includes: acquiring poster description text, which contains text information for adding to a poster; and selecting candidate background images that weakly match the text information from a pre-trained visual text model set as poster background images. Optionally, the instructions can also be executed by a processor of an electronic device to complete other steps involved in the exemplary embodiment described above.

[0249] Figure 10 This is an example diagram of an electronic device 500. Those skilled in the art will understand that it is illustrative. Figure 10 This is merely an example of electronic device 500 and does not constitute a limitation on electronic device 500. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 500 may also include input / output devices, network access devices, buses, etc.

[0250] The processor 502 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or processor 502 may be any conventional processor. Processor 502 is the control center of electronic device 500, connecting various parts of electronic device 500 through various interfaces and lines.

[0251] The memory 501 can be used to store computer-readable instructions 503. The processor 502 implements various functions of the electronic device 500 by running or executing the computer-readable instructions or modules stored in the memory 501 and calling the data stored in the memory 501. The memory 501 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 500, etc. In addition, the memory 501 may include a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0252] If the modules integrated in the electronic device 500 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, or it can instruct related hardware to complete the aforementioned computer-readable instructions, which can be stored in a computer-readable storage medium. When executed by a processor, these computer-readable instructions can implement the steps of the various method embodiments described above.

[0253] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0254] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of poster generation, characterized by, include: Obtain the poster description text, which contains text information to be added to the poster; Based on a pre-trained visual text model, candidate background images that are weakly correlated with the text information are selected from a pre-acquired set of candidate background images as poster background images. In the poster background image, determine the text layout area corresponding to the text information; The text information is filled into the text layout area to generate the target poster corresponding to the text information; The method further includes: Multiple candidate boxes are selected in the poster background image; and a salient image corresponding to the poster background image is generated using a spectral residual algorithm, wherein there are overlapping candidate boxes among the candidate boxes. Based on the salient average value of the salient image, the number of pixels of the candidate box, and a preset offset, a candidate value is determined for each candidate box. Candidate boxes with candidate values ​​lower than a preset candidate threshold are selected as target candidate boxes, and smooth regions in the poster background image are determined based on the target candidate boxes, wherein the positions of each target candidate box in the poster background image do not overlap; A usable area is selected in the smooth area using a first automatic encoder; The available area is sampled, and a text box corresponding to the available area is generated based on the corresponding sampling results and the text information to be added to the poster according to the poster description text, so as to obtain the target text layout area in the poster background image for writing the text information.

2. The poster generation method of claim 1, wherein, The pre-trained visual text model includes: BriVL containing a pre-trained image encoder and a pre-trained text encoder; The pre-trained visual text model selects candidate background images that are weakly correlated with the text information from a pre-acquired set of candidate background images as poster background images, including: Based on the pre-trained text encoder, the pre-trained text encoder is used to extract features from the text information to obtain text features; and, Based on the pre-trained image encoder, feature extraction is performed on each of the candidate background images to obtain the image features corresponding to each candidate background image; Calculate the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and select the candidate background image with the highest weak correlation feature similarity to the text features as the poster background image.

3. The poster generation method of claim 2, wherein, The pre-trained text encoder includes: the RoBERTa-Large encoder in the Chinese pre-trained model; The pre-trained text encoder, used to extract features from the text information to obtain text features, includes: The text information is input into the RoBERTa-Large encoder in the Chinese pre-trained model, so that the encoder can extract features from the text information and output the corresponding text features.

4. The poster generation method of claim 2, wherein, The pre-trained image encoders include: pre-trained Faster R-CNN and EfficientNet; The pre-trained image encoder extracts features from each candidate background image to obtain image features corresponding to each candidate background image, including: Each candidate background image is input into the pre-trained image encoder so that the pre-trained Faster R-CNN performs visual object detection processing on each candidate background image. Then, based on the EfficientNet, feature extraction is performed on each candidate background image after the visual object detection processing to obtain the image features corresponding to each candidate background image.

5. The poster generation method of claim 2, wherein, The step of calculating the weak correlation feature similarity between the text features and the image features corresponding to each of the candidate background images, and selecting the candidate background image with the highest weak correlation feature similarity to the text features to be compared as the poster background image, includes: The InfoNCE loss function, constructed based on a pre-defined weakly supervised learning method and a contrastive learning method CPC, determines the weakly correlated feature similarity between the text features and the image features corresponding to each of the candidate background images.

6. The poster generation method as described in claim 1, characterized in that, Determining the text layout area in the poster background image includes: The initial text layout area is determined by preliminary layout prediction of the poster background image using a first-level cascaded autoencoder; and... Based on the text length and text attributes represented by the text information, the initial text layout area is refined by a second-level cascaded autoencoder to obtain the text layout area.

7. The method of claim 1, wherein, The step of filling the text information into the text layout area to generate the poster corresponding to the text information includes: Extract the text features of the text information and detect the background color of the text layout area; Based on the background color of the text layout area, a target text style that matches the text features and the background color is selected from a preset text style database; The text information is converted according to the target text style and then filled into the text layout area to obtain the target poster corresponding to the text information.

8. An electronic device, comprising: include: Memory, used to store executable instructions; as well as, A processor, configured to work with the memory to execute the executable instructions to perform the operation of the method described in any one of claims 1-7.

9. A computer readable storage medium for storing computer readable instructions, characterized in that, When the instruction is executed, it performs the operation of any of the methods described in claims 1-7.

Images