Poster generation method and device and server

Abstract

The invention provides a poster generation method and device and a server. Based on the method, before specific implementation, an initial poster generation model comprising a plurality of initial modules and a plurality of objective functions about the plurality of initial modules can be constructed; and training the initial poster generation model by using the sample data based on the plurality of objective functions to obtain a preset poster generation model at least comprising a plurality of data processing modules including a foreground layer generation module, a background layer generation module, a layout control module, a layer fusion module and the like. In specific implementation, the method comprises the following steps: splicing acquired multi-modal data such as poster description text, initial image, poster layout mask and the like according to a preset splicing rule to obtain corresponding target combined data; and processing the target combined data by using a preset poster generation model to generate a target poster meeting the requirement. Therefore, the target poster which relates to a complex layer structure and is good in effect and high in quality can be efficiently, accurately and automatically generated.

Description

Technical Field

[0001] This manual belongs to the field of Internet technology, and in particular relates to methods, devices and servers for generating posters. Background Technology

[0002] In financial business scenarios, financial institutions often need to create corresponding promotional posters when organizing online and offline promotional activities in order to attract customer attention and participation.

[0003] However, existing methods can only automatically generate posters with relatively simple structures using existing models. Furthermore, posters generated using these methods are mostly of poor quality and unsatisfactory, requiring manual adjustments by technical personnel.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides a poster generation method, apparatus, and server that can efficiently and accurately automatically generate target posters with complex layer structures, resulting in good effects and high quality.

[0006] This manual provides a method for generating posters, including:

[0007] Get the poster description text, initial image, and poster layout mask;

[0008] The poster description text, initial image, and poster layout mask are spliced ​​together according to preset splicing rules to obtain the corresponding target combination data;

[0009] By processing the target combination data using a preset poster generation model, a target poster that meets the requirements is generated; wherein, the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and, the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

[0010] In one embodiment, a preset poster generation model is used to process the target combination data to generate a target poster that meets the requirements, including:

[0011] The foreground layer generation module processes the poster description text and initial image in the target combination data to generate the initial foreground layer.

[0012] The second adjustment unit is used to obtain the consistency features of the initial foreground layer through the first adjustment unit;

[0013] The second adjustment unit uses the consistency feature to control the background layer generation module to process the poster description text and initial image in the target combination data, and generates an initial background layer that matches the initial foreground layer.

[0014] The layout control module determines the relative layout information of the foreground layer and the background layer based on the initial foreground layer, the initial background layer, and the poster layout mask in the target combination data.

[0015] The first adjustment unit controls the foreground layer generation module based on relative layout information to fine-tune the initial foreground layer, thereby obtaining the corresponding target foreground layer; the second adjustment unit controls the background generation module based on relative layout information to fine-tune the initial background layer, thereby obtaining the corresponding target background layer.

[0016] The layer blending module is used to blend and stitch the target foreground layer and the target background layer according to the relative layout information to obtain a target poster that meets the requirements.

[0017] In one embodiment, the target foreground layer includes multiple foreground layers; wherein each of the multiple foreground layers is provided with a corresponding priority label.

[0018] In one embodiment, the foreground layer generation module is further equipped with a text encoder;

[0019] Accordingly, the process of using the foreground layer generation module to process the poster description text and initial image in the target combination data to generate an initial foreground layer includes:

[0020] The poster description text is converted into a corresponding text vector using a text encoder.

[0021] The foreground layer generation module extracts global text features by processing text vectors; the foreground layer generation module also extracts global image features by processing the initial image.

[0022] The first adjustment unit determines the global style type of the poster based on the global text features and / or global image features; and determines the target feature mapping rule that matches the global style type of the poster from the preset feature mapping rule set.

[0023] The first adjustment unit controls the foreground layer generation module to process the text vector and the initial image based on the target feature mapping rules to generate an initial foreground layer.

[0024] In one embodiment, after using the foreground layer generation module to process text vectors and extract global text features, and using the foreground layer generation module to process the initial image and extract global image features, the method further includes:

[0025] The first adjustment unit uses the global text features and / or global image features to determine whether local precision control is needed when generating the foreground layer;

[0026] When it is determined that local precision control is required, the first adjustment unit determines the precision enhancement area in the foreground layer based on the global text features and / or global image features;

[0027] The first adjustment unit is used to adjust the feature weights of the foreground layer generation module for the precision enhancement area;

[0028] The adjusted foreground layer generation module is used to process the text vector and the initial image to generate an initial foreground layer.

[0029] In one embodiment, the preset poster generation model is trained in the following manner:

[0030] Construct an initial foreground layer generation module, an initial background layer generation module, an initial layout control module, and an initial layer blending module; wherein, the initial foreground layer generation module deploys a first adjustment unit, and the initial background layer generation module deploys a second adjustment unit;

[0031] The initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module are combined; and the first adjustment unit and the second adjustment unit are connected using a specified data channel to obtain the initial poster generation model.

[0032] Obtain sample data that meets the requirements; and construct multiple objective functions for the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module;

[0033] Based on the multiple objective functions, the initial poster generation model is trained using the sample data to obtain a preset poster generation model that meets the requirements.

[0034] In one embodiment, obtaining the required sample data includes:

[0035] Obtain sample poster images;

[0036] Based on the sample poster image, the corresponding sample foreground layer and sample background layer are obtained by splitting it;

[0037] Based on the sample poster image, sample foreground layer, and sample background layer, a sample layout mask for the sample poster image is determined.

[0038] By combining the sample poster image, sample foreground layer, sample background layer, and sample layout mask, sample data that meets the requirements is obtained.

[0039] In one embodiment, the plurality of objective functions include a first objective function, a second objective function, a third objective function, and a fourth objective function corresponding to the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer fusion module in the initial poster generation model, respectively; wherein, different objective functions have different loss constraints.

[0040] In one embodiment, a first objective function corresponding to the initial foreground layer generation module is constructed according to the following formula:

[0041]

[0042] in, The value of the first objective function. Indicates based on arrive Expectations The poster image generated at the current time point, where t is the current time point. The poster image generated at the initial time point. The foreground layer generated by the module for the current time point is the output foreground layer. For transparency loss, The opacity channel data value of the foreground layer. For the purpose of transparency, The weighting coefficients for the transparency loss term. This is the transparency channel.

[0043] In one embodiment, a second objective function corresponding to the initial background layer generation module is constructed according to the following formula:

[0044]

[0045] in, The value of the second objective function. The background layer output by the module that generates the background image for the current time point. For style matching loss, This represents the function for calculating the style matrix. The weighting coefficients for the style matching loss term. For consistency loss items, This represents the consistency feature extraction function. The weighting coefficients for the consistency loss term.

[0046] In one embodiment, training the initial poster generation model using the sample data based on the plurality of objective functions includes:

[0047] During model training, the first adjustment unit and the second adjustment unit are used to dynamically adjust the first noise addition rate control parameter of the foreground layer generation module and the second noise addition rate control parameter of the background layer generation module based on the cumulative duration between the current time point and the initial time point, and / or the layer area of ​​interest at the current time point.

[0048] This specification also provides a poster generating apparatus, including:

[0049] The acquisition module is used to acquire the poster description text, initial image, and poster layout mask;

[0050] The splicing module is used to splice the poster description text, initial image, and poster layout mask according to preset splicing rules to obtain the corresponding target combination data;

[0051] The generation module is used to generate a target poster that meets the requirements by processing the target combination data using a preset poster generation model; wherein the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

[0052] This specification also provides a server, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the steps of the poster generation method.

[0053] This specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the relevant steps of the poster generation method.

[0054] This specification also provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the poster generation method.

[0055] Based on the poster generation method, apparatus, and server provided in this specification, before implementation, an initial poster generation model containing multiple initial modules can be constructed. Simultaneously, multiple objective functions corresponding to the initial modules in the initial poster generation model are constructed. Then, based on the multiple objective functions, the initial poster generation model is trained using sample data to obtain a preset poster generation model that meets the requirements. The preset poster generation model includes at least several data processing modules: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module. Furthermore, the foreground layer generation module is equipped with a first adjustment unit, and the background layer generation module is equipped with a second adjustment unit, with the first and second adjustment units connected. In specific implementation, the acquired poster description text, initial image, poster layout mask, and other multimodal data can be spliced ​​according to preset splicing rules to obtain corresponding target combination data. Then, the preset poster generation model is used to process the aforementioned target combination data to generate a target poster that meets the requirements. This allows for the efficient and accurate automatic generation of target posters with complex layer structures, resulting in better and higher quality posters. It ensures that the content of the generated target posters is harmonious and consistent, eliminating the need for manual corrections by technical personnel and effectively reducing their workload. Attached Figure Description

[0056] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating a poster generation method provided in one embodiment of this specification;

[0058] Figure 2 This is a schematic diagram illustrating one embodiment of the poster generation method provided in this specification, applied in a scenario example.

[0059] Figure 3 This is a schematic diagram illustrating one embodiment of the poster generation method provided in this specification, applied in a scenario example.

[0060] Figure 4 This is a schematic diagram illustrating one embodiment of the poster generation method provided in this specification, applied in a scenario example.

[0061] Figure 5 This is a schematic diagram illustrating one embodiment of the poster generation method provided in this specification, applied in a scenario example.

[0062] Figure 6 This is a schematic diagram illustrating one embodiment of the poster generation method provided in this specification, applied in a scenario example.

[0063] Figure 7 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0064] Figure 8 This is a schematic diagram of the structural composition of a poster generation device provided in one embodiment of this specification. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0066] It should be noted that the information and data related to users involved in the embodiments of this specification are all information and data authorized by the user or fully authorized by the relevant parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, and necessary confidentiality measures have been taken. They do not violate public order and good morals, and corresponding operation entry points are provided for users or relevant parties to choose to authorize or refuse.

[0067] It should also be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0068] See Figure 1 As shown in the embodiments of this specification, a method for generating a poster is provided. In specific implementation, the method may include the following:

[0069] S101: Obtain poster description text, initial image, and poster layout mask;

[0070] S102: According to the preset splicing rules, splice the poster description text, initial image, and poster layout mask to obtain the corresponding target combination data;

[0071] S103: Using a preset poster generation model, a target poster that meets the requirements is generated by processing the target combination data; wherein, the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and, the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

[0072] Specifically, the method for generating the aforementioned poster can be applied to the server side.

[0073] The aforementioned server may specifically include a backend server responsible for data processing, applied to one side of a business platform (e.g., a poster intelligent generation platform), capable of data transmission, data processing, and other functions. Specifically, the server may be, for example, an electronic device with data computing, storage, and network interaction capabilities. Alternatively, the server may be a software program running on the electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server may be a single server, several servers, or a server cluster formed by several servers.

[0074] In practice, when a user needs to generate a poster, they can send a target poster generation request to the platform's server via their terminal. This triggers the server to apply the poster generation method and automatically generate a target poster that meets the user's requirements. The target poster generation request may include at least poster description text, an initial image, and a poster layout mask.

[0075] The aforementioned user terminal may specifically include a front-end applied to the user side, capable of data collection, data transmission, and other functions. Specifically, the user terminal may be an electronic device such as a desktop computer, tablet computer, laptop computer, or smartphone. Alternatively, the user terminal may also be a software application that can run on the aforementioned electronic device. For example, it could be a smart poster app running on a smartphone.

[0076] In practice, users can input poster description text, an initial image, and a poster layout mask through a user terminal. Correspondingly, the user terminal can receive and use this data to generate a target poster generation request, which includes at least the poster description text, the initial image, and the poster layout mask; and then send this target poster generation request to the server. The server receives the target poster generation request and, based on it, obtains the poster description text, the initial image, and the poster layout mask.

[0077] Specifically, the poster description text mentioned above can be understood as text data used to describe the user's requirements for the target poster to be generated, such as content, format, layout, and images.

[0078] The initial image mentioned above can be understood as the base image that the target poster to be generated needs to emphasize or refer to. Specifically, the initial image can be a simplified diagram containing only the subject object (e.g., the object of the promotional event or the product object).

[0079] The aforementioned poster layout mask can be understood as data used to characterize the layout information of the target poster to be generated. Specifically, for example, the aforementioned poster layout mask can be a binary image (or grayscale image), in which a first specific pixel value (e.g., 255) can be used to identify the foreground area in the poster, and a second specific pixel value (e.g., 0) can be used to identify the background area in the poster.

[0080] The aforementioned preset poster generation model can be understood as a hybrid structure based on diffusion model and neural network model, containing multiple specific data processing modules, and is an algorithm model that can automatically generate posters involving complex layer structures.

[0081] The aforementioned diffusion model can be understood as an algorithmic model that generates results by progressively adding noise.

[0082] For details, please refer to Figure 2 As shown, the preset poster generation model may include at least the following structures: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module.

[0083] The foreground layer generation module is equipped with a first adjustment unit, and the background layer generation module is equipped with a second adjustment unit. The first adjustment unit and the second adjustment unit can be connected through a specified data channel so that the first adjustment unit and the second adjustment unit can interact with each other on information related to the foreground layer and the background layer.

[0084] Specifically, the first and second adjustment units mentioned above can be adjustment control structures based on low-rank matrices. Accordingly, based on the first and second adjustment units, during the operation of the preset poster generation model, the operation of the foreground layer generation module and the background layer generation module can be dynamically adjusted and controlled according to specific situations and needs, so as to obtain foreground and background layers that are relatively harmonious and consistent in overall image content and style.

[0085] Further reading Figure 2As shown, the first adjustment unit and the second adjustment unit are also connected to the layout control module. Accordingly, based on the first adjustment unit and the second adjustment unit, during the operation of the preset poster generation model, the foreground layer generation module and the background layer generation module can be intelligently controlled to fine-tune the previously output foreground layer and background layer according to the relative layout information output by the layout control module, so as to obtain a foreground layer and background layer that are relatively harmonious and consistent in the overall image layout.

[0086] Specifically, the first adjustment unit and the second adjustment unit include a LoRA structure based on a low-rank matrix.

[0087] The aforementioned layer blending module is also connected to the foreground layer generation module, the background layer generation module, and the layout control module, respectively. Accordingly, based on the layer blending module, it can receive and utilize the data information output by the foreground layer generation module, the background layer generation module, and the layout control module, and through blending and stitching, obtain the target poster that meets the requirements.

[0088] Specifically, the aforementioned target poster can be understood as a promotional poster that is effective, of high quality, and suitable for direct application deployment. In particular, the target poster can include posters tailored to different deployment scenarios.

[0089] The aforementioned advertising scenarios may include at least one of the following: promotion of financial products, advertising of financial service activities, and notification of financial procedures and rules, etc. It should be noted that the advertising scenarios listed above are merely illustrative. In actual implementation, depending on the specific circumstances and processing needs, the target posters mentioned above may also be promotional posters for other suitable types of advertising scenarios. This specification does not limit this.

[0090] Furthermore, the aforementioned target poster may also include posters involving multiple layers, with differences in the detailed textures of objects in different layers, resulting in a more complex layer structure.

[0091] In practice, the initial image and poster layout mask mentioned above can be empty in the target poster generation request. Accordingly, after receiving the target poster generation request, the server can first obtain and extract the image requirements and layout requirements based on the poster description text; then, according to the preset prompting rules, the image requirements and layout requirements are converted into corresponding image prompt words and layout feature words; and finally, a pre-trained large language model is used to generate the corresponding initial image and poster layout mask based on the image prompt words and layout prompt words.

[0092] In practice, after obtaining the poster description text, initial image, and poster layout mask, the poster description text, initial image, and poster layout mask can be preprocessed accordingly.

[0093] Specifically, for the poster description text, a semantic processing model based on NLP (Natural Language Processing) can be used to perform word order detection and semantic error detection on the poster description text, obtaining the corresponding word order detection results and semantic error detection results. Then, based on the word order detection results, the word order in the poster description text is adjusted (e.g., inverted word order), resulting in a word order-adjusted poster description text adapted to the preset poster generation model. Further, based on the semantic error detection results, text units with obvious semantic errors are identified in the word order-adjusted poster description text, and these text units are automatically corrected to obtain the preprocessed poster description text.

[0094] For the initial image, a preset filtering model can be used to filter the initial image to eliminate image noise and obtain a denoised initial image; then the contrast and sharpness of the denoised initial image are adjusted to enhance the subject object in the initial image, resulting in a preprocessed initial image.

[0095] For poster layout masks, the mask format can be detected based on the preset layout mask template to obtain the mask format detection result; based on the mask format detection result, the data format of the poster layout mask can be adaptively adjusted to obtain a preprocessed poster layout mask that is compatible with the preset poster generation model.

[0096] In practice, the poster description text, initial image, and poster layout mask can be combined in a specified order according to the preset splicing rules to obtain the target combination data that can be successfully recognized and processed by the subsequent preset poster generation model.

[0097] In practice, multiple data processing modules in the aforementioned preset poster generation model can be used to process the input target combination data to ultimately generate and output a target poster that meets the requirements.

[0098] Based on the above embodiments, a preset poster generation model that utilizes a hybrid structure based on diffusion and neural network models and includes multiple interconnected data processing modules can efficiently and accurately generate target posters with complex layer structures, good effects, and high quality that can be directly applied to applications. This eliminates the need for manual adjustments by relevant technical personnel, effectively reducing their workload and improving overall data processing efficiency.

[0099] In some embodiments, see Figure 3As shown, the above-described poster generation model uses a preset poster generation model to process the target combination data and generate a target poster that meets the requirements. In specific implementation, it may include the following:

[0100] S1: Use the foreground layer generation module to process the poster description text and initial image in the target combination data to generate the initial foreground layer;

[0101] S2: Using the second adjustment unit through the first adjustment unit, obtain the consistency features of the initial foreground layer;

[0102] S3: The second adjustment unit controls the background layer generation module to process the poster description text and initial image in the target combination data based on the consistency feature, and generates an initial background layer that matches the initial foreground layer.

[0103] S4: Using the layout control module, determine the relative layout information of the foreground layer and the background layer based on the initial foreground layer, the initial background layer, and the poster layout mask in the target combination data.

[0104] S5: The first adjustment unit controls the foreground layer generation module based on relative layout information to fine-tune the initial foreground layer to obtain the corresponding target foreground layer; the second adjustment unit controls the background generation module based on relative layout information to fine-tune the initial background layer to obtain the corresponding target background layer.

[0105] S6: Using the layer blending module, the target foreground layer and the target background layer are blended and spliced ​​according to the relative layout information to obtain a target poster that meets the requirements.

[0106] Specifically, the aforementioned consistency features may include at least one of the following: illumination consistency features, content consistency features, logical consistency features, etc.

[0107] It should be noted that the consistency features listed above are merely illustrative. In actual implementation, other suitable consistency features may be included depending on the specific circumstances and processing requirements. This specification does not limit this.

[0108] Based on the above embodiments, the model structure of the preset poster generation model can be fully utilized. By combining and coordinating multiple data processing modules in the preset poster generation model, target posters with complex layer structures, good effects, and high quality can be generated automatically, efficiently and accurately.

[0109] In some embodiments, the target foreground layer may specifically include multiple foreground layers; wherein, each of the multiple foreground layers is provided with a corresponding priority label.

[0110] The aforementioned multiple foreground layers may specifically include: layers containing different subject objects and / or layers containing different detailed texture areas.

[0111] The priority labels mentioned above specifically indicate the level of detail required for the foreground layer during the generation process. These priority labels can be determined based on the importance of the subject object and / or the complexity of the detailed texture areas within the foreground layer.

[0112] In specific implementation, after the foreground layer generation module processes the poster description text and initial image in the target combination data to generate the initial foreground layer, the first adjustment unit can first detect whether the number of initial foreground layers is greater than 1. If it is determined to be greater than 1, that is, the initial foreground layer includes multiple initial foreground layers, the first adjustment unit can send the corresponding foreground layer trigger signal to the preset poster generation model. The preset poster generation model responds to the foreground layer trigger signal, determines the priority labels of multiple initial foreground layers based on the poster description text and initial image, and applies the corresponding priority labels to multiple initial foreground layers. Then, the first adjustment unit can detect the priority labels of the initial foreground layers, determine the initial foreground layers with priority labels greater than the preset priority level, that is, the initial foreground layers with high precision requirements; then, based on the priority labels, the first adjustment unit controls the foreground layer generation module to further refine the initial foreground layers with priority labels greater than the preset priority level on the basis of the original initial foreground layers, so as to enhance the subject object in the layer and / or improve the resolution of the detailed texture area, etc., to obtain an initial foreground layer that meets the requirements.

[0113] Furthermore, the aforementioned target background layer may also include multiple background layers; wherein, each of the multiple background layers may also be assigned a corresponding priority label. Similarly, the second adjustment unit and the background layer generation module can further refine the initial background layers whose priority labels are greater than a preset priority level, based on the priority labels.

[0114] Based on the above embodiments, different foreground layers can be differentiated by introducing and using priority tags, which can better adapt to situations with complex layer structures and generate multiple high-quality target foreground layers.

[0115] In some embodiments, see Figure 2 As shown, the foreground layer generation module may also include a text encoder.

[0116] Accordingly, see Figure 4As shown, the foreground layer generation module processes the poster description text and initial image in the target combination data to generate an initial foreground layer. In specific implementation, this may include the following:

[0117] S1: Use a text encoder to convert the poster description text into a corresponding text vector;

[0118] S2: The foreground layer generation module extracts global text features by processing text vectors; the foreground layer generation module extracts global image features by processing the initial image.

[0119] S3: The first adjustment unit determines the global style type of the poster based on the global text features and / or global image features; and determines the target feature mapping rule that matches the global style type of the poster from the preset feature mapping rule set.

[0120] S4: The first adjustment unit controls the foreground layer generation module to process the text vector and the initial image based on the target feature mapping rules to generate an initial foreground layer.

[0121] Specifically, the aforementioned global style type may include at least one of the following: hand-drawn style, realistic style, artistic style, popular style, etc.

[0122] It should be noted that the global style types listed above are merely illustrative. In practice, other suitable global style types may be included depending on the specific circumstances and processing requirements. This specification does not limit this.

[0123] The aforementioned preset feature mapping rule set may include multiple preset feature mapping rules. Each preset feature mapping rule corresponds to at least one global style type.

[0124] Before implementation, a large number of sample poster images can be obtained; then, based on the global style type of the sample poster images, cluster learning is performed using the sample poster images to determine multiple preset feature mapping rules corresponding to each global style type; and the multiple preset feature mapping rules are combined to construct a preset feature mapping rule set.

[0125] Accordingly, in specific implementation, the first adjustment unit can also send the global style type of the poster to the second adjustment unit. The second adjustment unit can determine the target feature mapping rule that matches the global style type of the poster from the preset feature mapping rule set. Then, the second adjustment unit can use the consistency feature to control the background layer generation module to process the poster description text and initial image in the target combination data based on the target feature mapping rule, and generate an initial background layer that matches the initial foreground layer and has a harmonious and unified style.

[0126] Based on the above embodiments, by using the first adjustment unit to determine and control the operation of the foreground layer generation module according to the global style type of the poster, an initial foreground layer that meets the user's style type can be accurately and automatically generated.

[0127] In some embodiments, after the foreground layer generation module processes text vectors to extract global text features and processes the initial image to extract global image features, refer to [reference needed]. Figure 5 As shown, in specific implementations, the method may also include the following:

[0128] S1: The first adjustment unit determines whether local precision control is needed when generating the foreground layer based on the global text features and / or global image features;

[0129] S2: When it is determined that local precision control is needed, the first adjustment unit determines the precision enhancement area in the foreground layer based on the global text features and / or global image features;

[0130] S3: Use the first adjustment unit to adjust the feature weights of the foreground layer generation module for the precision enhancement area;

[0131] S4: Using the adjusted foreground layer generation module, process the text vector and the initial image to generate an initial foreground layer.

[0132] The aforementioned precision enhancement area can be an area containing the subject object, and / or a detailed texture area (e.g., a more complex cutout area).

[0133] Specifically, the aforementioned detailed texture areas may include one or more of the following: cutout areas, complex detail areas, fine texture areas, etc.

[0134] In practice, for areas requiring enhanced precision, the first adjustment unit can increase the feature weights for these areas, allowing the foreground layer generation module to spend more time on these areas and perform more refined processing, resulting in an initial foreground layer with higher precision and richer details. Conversely, for areas without enhanced precision, the first adjustment unit can use the default feature weights to control the foreground layer to undergo relatively conventional processing, generating the initial foreground layer quickly with shorter processing time.

[0135] In practice, considering that the background layer is often less important than the foreground layer in a poster, the second adjustment unit can be used to adjust the feature weights of the background layer generation module for the background layer. This allows the background layer generation module to process the poster description text and initial image in the target combination data with a smaller feature weight than the foreground layer, thus quickly generating the corresponding initial background layer.

[0136] In some embodiments, as specifically implemented, refer to Figure 6 As shown, the preset poster generation model can be trained in the following way:

[0137] S1: Construct the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module; wherein, the initial foreground layer generation module is equipped with a first adjustment unit, and the initial background layer generation module is equipped with a second adjustment unit;

[0138] S2: Combine the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module; and connect the first adjustment unit and the second adjustment unit using a specified data channel to obtain the initial poster generation model;

[0139] S3: Obtain sample data that meets the requirements; and construct multiple objective functions for the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module;

[0140] S4: Based on the multiple objective functions, train the initial poster generation model using the sample data to obtain a preset poster generation model that meets the requirements.

[0141] Specifically, the initial poster generation model mentioned above can be an initial model constructed based on a hybrid structure of diffusion model and neural network model.

[0142] In practice, during the initial poster generation model training process using sample data, the aforementioned multiple objective functions can be used to calculate and modify the corresponding network parameters (for example, they can be denoted as...). The model is continuously adjusted and updated until the values ​​of multiple objective functions calculated based on the above multiple objective functions meet the requirements (e.g., reach the optimal value), thereby determining multiple data processing modules that meet the requirements, and thus obtaining a preset poster generation model that meets the requirements.

[0143] Based on the above embodiments, a preset poster generation model that meets the requirements can be trained by constructing and utilizing an initial poster generation model that integrates a diffusion model and a neural network model.

[0144] In some embodiments, obtaining qualified sample data may include the following:

[0145] S1: Obtain the sample poster image;

[0146] S2: Based on the sample poster image, the corresponding sample foreground layer and sample background layer are obtained by splitting it;

[0147] S3: Determine the sample layout mask for the sample poster image based on the sample poster image, the sample foreground layer, and the sample background layer;

[0148] S4: Combine the sample poster image, sample foreground layer, sample background layer, and sample layout mask to obtain sample data that meets the requirements.

[0149] In practice, after obtaining the sample poster images, the poster quality can be tested based on multiple factors such as the poster's promotional effect, image clarity, and information completeness. The poster quality test results are then obtained. Based on the poster quality test results, sample poster images with poor quality (e.g., below a preset quality threshold) are removed from the sample poster images.

[0150] Based on the above embodiments, high-quality and effective sample data can be constructed using sample poster images.

[0151] In some embodiments, the plurality of objective functions include a first objective function, a second objective function, a third objective function, and a fourth objective function corresponding to the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer fusion module in the initial poster generation model, respectively; wherein, different objective functions have different loss constraints.

[0152] In some embodiments, in specific implementations, a first objective function corresponding to the initial foreground layer generation module can be constructed according to the following formula:

[0153]

[0154] in, The value of the first objective function. Indicates based on arrive Expectations The poster image generated at the current time point, where t is the current time point. The poster image generated at the initial time point. The foreground layer generated by the module for the current time point is the output foreground layer. For transparency loss, The opacity channel data value of the foreground layer. For the purpose of transparency, The weighting coefficients for the transparency loss term. This refers to the transparency channel. Specifically, "foreground" represents the foreground layer generation module. Represents the network parameters of the model. This represents the absolute value of the difference between the foreground layer output by the foreground layer generation module at the current time point and the foreground layer in the poster image generated at the initial time point.

[0155] In practice, based on the first objective function mentioned above, by introducing a transparency loss term, the foreground layer generated by the foreground layer generation module in the model can have the expected transparency, thereby enabling clearer and richer image details to be displayed in some complex areas.

[0156] In some embodiments, a second objective function corresponding to the initial background layer generation module can be constructed according to the following formula:

[0157]

[0158] in, The value of the second objective function. The background layer output by the module that generates the background image for the current time point. For style matching loss, This represents the function for calculating the style matrix. The weighting coefficients for the style matching loss term. For consistency loss items, This represents the consistency feature extraction function. These are the weighting coefficients for the consistency loss term. Specifically, "background" represents the background layer generation module. Represents the network parameters of the model. This represents the absolute value of the difference between the background layer output by the background layer generation module at the current time point and the background layer in the poster image generated at the initial time point.

[0159] In practice, based on the second objective function mentioned above, by introducing style matching loss term and consistency loss term, the background layer generated by the background layer generation module in the model can have high style matching and feature consistency with the foreground layer generated by the previous foreground layer generation module, so that the visual attributes of the background and foreground are relatively unified.

[0160] In some embodiments, a third objective function corresponding to the initial layout control module can be constructed according to the following formula:

[0161]

[0162] in, The value of the third objective function. The expectation is based on m, where m is the data value of the layout mask. To lay out a mask for the target, For geometric constraint functions, This is a spatial constraint loss term relating to the relative positions of the foreground and background. As the first weighting coefficient, This is the second weighting coefficient. Let i be the local component numbered i in the layout mask. For the target layout mask, select the local component numbered i. This refers to the local dynamic weight numbered i. Specifically, layout represents the layout control module, and the specific value of the aforementioned local dynamic weight can be dynamically determined based on the local complexity (e.g., the proportion of foreground in the local area, the complexity of the shape of the local area, etc.).

[0163] In practice, based on the third objective function mentioned above, a spatial constraint loss term is introduced to enable the layout control module in the model to impose more effective geometric constraints on the relative positional relationship between the foreground and the background (e.g., distance, alignment loss, etc.) to ensure that the overall layout of the foreground and the background is more in line with the visual positional rationality.

[0164] In some embodiments, in specific implementations, the fourth objective function corresponding to the initial layer fusion module can be constructed according to the following formula:

[0165]

[0166] in, The value of the fourth objective function. Indicates based on arrive Expectations The foreground layer generated by the module for the current time point is the output foreground layer. The background layer output by the module that generates the background image for the current time point. For blending operators that include transparency blending and edge smoothing, For perceptual feature extraction function, To fuse the perceived loss term, These are the weighting coefficients for perceived loss. Specifically, fuse represents the layer blending module. This represents the absolute value of the difference between the stitched result of the background layer output by the background layer generation module at the current time point and the foreground layer output by the foreground layer generation module at the current time point, and the poster image generated at the initial time point.

[0167] In practice, based on the fourth objective function mentioned above, the fusion operator and fusion perception loss term are used to make the fusion and stitching of the background layer and the foreground layer in the model not a simple pixel superposition, but based on the VGG feature distance, so that the fusion result is more natural and harmonious in terms of semantics and visual perception, and closer to the poster image drawn by hand.

[0168] In some embodiments, the initial poster generation model is trained using the sample data based on the plurality of objective functions. In specific implementations, this may further include the following:

[0169] During model training, the first adjustment unit and the second adjustment unit are used to dynamically adjust the first noise addition rate control parameter of the foreground layer generation module and the second noise addition rate control parameter of the background layer generation module based on the cumulative duration between the current time point and the initial time point, and / or the layer area of ​​interest at the current time point.

[0170] Specifically, as the cumulative training time increases, the first noise addition rate control parameter of the foreground layer generation module and the background layer generation module can be gradually increased. This way, in the early stages of model training, only a small amount of noise is provided, allowing the relevant data processing modules in the model to focus more on learning the basic rules of the poster image. As the model continues to train and improve, the noise interference is continuously increased by dynamically increasing the first noise addition rate control parameter, thereby increasing the difficulty of model training. In the later stages of model training, after the model has learned and mastered a relatively complete set of basic rules, the focus shifts to learning and processing relevant noise interference in the poster image, thereby improving the model's robustness and generalization ability, enabling the model to better cope with and handle poster images in complex situations.

[0171] Specifically, considering that the foreground layer is often more important than the background layer in a poster image, during model training, a relatively large amount of second noise can be added as a rate control parameter for the foreground layer generation module, and more noise can be used for training to enable the foreground layer generation module to learn and process the foreground layer more precisely. Conversely, a relatively small amount of second noise can be added as a rate control parameter for the background layer generation module, and less noise can be used for training to enable the background layer generation module to learn and process the background layer more quickly.

[0172] Furthermore, during training, it is possible to distinguish whether the image regions processed by the foreground layer generation module and the background layer generation module at the current time point belong to the layer regions of interest (e.g., high-importance precision enhancement areas). For the layer regions of interest, the second noise additive control parameter at the current time point can be dynamically increased to focus on learning the layer regions of interest at the current time point; conversely, for the layer regions of non-interest, the second noise additive control parameter can be dynamically decreased to quickly and relatively simply learn the layer regions of non-interest at the current time point.

[0173] In some embodiments, when specifically training the initial wave generation model, the initial foreground layer generation module can also perform corresponding specialized reinforcement training for different types of detail texture regions (e.g., hollow areas, complex detail areas, fine texture areas, etc.) to improve the data processing performance of the foreground layer generation module.

[0174] In some embodiments, after generating a target poster that meets the requirements, the method may further include the following: determining the placement scenario of the target poster; determining a matching target publicity risk detection model from a preset poster publicity risk model library based on the placement scenario; processing the target poster using the target publicity detection model to obtain the corresponding target risk detection result; and determining whether the target poster has publicity risk based on the target risk detection result.

[0175] Based on the target risk detection results, when it is determined that the target poster has a publicity risk, the target poster can be corrected and updated according to the target risk detection results to obtain the updated target poster; then the updated target poster can be released to the public.

[0176] Specifically, for example, in the context of promoting financial products, the aforementioned target advertising risk detection model can detect whether the target poster contains non-compliant financial promotions or risks of disclosing users' personal information. Accordingly, based on the target risk detection results, these risks in the target poster can be identified and addressed in a timely and effective manner to eliminate them.

[0177] Conversely, if the target poster is determined to have no publicity risk based on the target risk detection results, it can be released directly to the public.

[0178] As can be seen from the above, based on the poster generation method provided in the embodiments of this specification, before specific implementation, an initial poster generation model containing multiple initial modules can be constructed; simultaneously, multiple objective functions for the multiple initial modules are constructed; then, based on the multiple objective functions, the initial poster generation model is trained using sample data to obtain a preset poster generation model that meets the requirements; wherein, the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and, the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected. In specific implementation, the obtained poster description text, initial image, poster layout mask, and other multimodal data can be spliced ​​according to preset splicing rules to obtain corresponding target combination data; then, the preset poster generation model is used to process the above target combination data to generate a target poster that meets the requirements. Thus, it is possible to efficiently and accurately automatically generate target posters involving complex layer structures with good effects and high quality, effectively reducing the workload of relevant technical personnel.

[0179] This specification provides an embodiment of a server, see below. Figure 7 As shown. The server includes a network communication port 701, a processor 702, and a memory 703. These structures are connected by internal cables so that they can perform specific data interaction.

[0180] Specifically, the network communication port 701 can be used to acquire poster description text, initial image, and poster layout mask.

[0181] The processor 702 is specifically used to splice the poster description text, initial image, and poster layout mask according to preset splicing rules to obtain corresponding target combination data; and to generate a target poster that meets the requirements by processing the target combination data using a preset poster generation model; wherein the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit is connected to the second adjustment unit.

[0182] The memory 703 can be used to store the corresponding instruction program and related intermediate data.

[0183] Based on the above method, the relevant structural performance of the server can be effectively utilized to improve the data processing speed of electronic devices and efficiently realize the data processing for poster generation.

[0184] In this embodiment, the network communication port 701 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0185] In this embodiment, the processor 702 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0186] In this embodiment, the memory 703 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0187] This specification also provides a computer-readable storage medium based on the above-described poster generation method. The computer-readable storage medium stores computer program instructions that, when executed, perform the following: acquiring poster description text, an initial image, and a poster layout mask; splicing the poster description text, the initial image, and the poster layout mask according to preset splicing rules to obtain corresponding target combination data; and generating a target poster that meets the requirements by processing the target combination data using a preset poster generation model. The preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module. Furthermore, the foreground layer generation module is equipped with a first adjustment unit, and the background layer generation module is equipped with a second adjustment unit, with the first adjustment unit connected to the second adjustment unit.

[0188] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0189] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0190] This specification also provides a computer program product, comprising at least a computer program, which, when executed by a processor, implements the following method steps: acquiring poster description text, an initial image, and a poster layout mask; splicing the poster description text, the initial image, and the poster layout mask according to preset splicing rules to obtain corresponding target combination data; and generating a target poster that meets the requirements by processing the target combination data using a preset poster generation model; wherein the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

[0191] See Figure 8 As shown in the embodiments of this specification, a poster generation apparatus is also provided, which may specifically include the following structural modules:

[0192] The acquisition module 801 can be used to acquire poster description text, initial image, and poster layout mask;

[0193] The splicing module 802 can be used to splice the poster description text, initial image, and poster layout mask according to preset splicing rules to obtain the corresponding target combination data;

[0194] The generation module 803 is specifically used to generate a target poster that meets the requirements by processing the target combination data using a preset poster generation model; wherein the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit is connected to the second adjustment unit.

[0195] In some embodiments, when the above-mentioned generation module 803 is specifically implemented, it can generate a target poster that meets the requirements by processing the target combination data using a preset poster generation model in the following manner: The foreground layer generation module processes the poster description text and initial image in the target combination data to generate an initial foreground layer; the second adjustment unit obtains the consistency features of the initial foreground layer through the first adjustment unit; the second adjustment unit controls the background layer generation module to process the poster description text and initial image in the target combination data based on the consistency features to generate an initial background layer that matches the initial foreground layer; the layout control module determines the relative layout information of the foreground layer and the background layer based on the initial foreground layer, the initial background layer, and the poster layout mask in the target combination data; the first adjustment unit controls the foreground layer generation module to fine-tune the initial foreground layer based on the relative layout information to obtain a corresponding target foreground layer; the second adjustment unit controls the background generation module to fine-tune the initial background layer based on the relative layout information to obtain a corresponding target background layer; and the layer fusion module fuses and splices the target foreground layer and the target background layer based on the relative layout information to obtain a target poster that meets the requirements.

[0196] In some embodiments, the target foreground layer may specifically include multiple foreground layers; wherein, each of the multiple foreground layers is provided with a corresponding priority label.

[0197] In some embodiments, the foreground layer generation module may further include a text encoder;

[0198] Accordingly, in specific implementation, the foreground layer generation module 803 can process the poster description text and initial image in the target combination data using the foreground layer generation module in the following manner to generate an initial foreground layer: The poster description text is converted into a corresponding text vector using a text encoder; global text features are extracted by processing the text vector using the foreground layer generation module; global image features are extracted by processing the initial image using the foreground layer generation module; the global style type of the poster is determined by the first adjustment unit based on the global text features and / or global image features; and a target feature mapping rule matching the global style type of the poster is determined from a preset feature mapping rule set; the foreground layer generation module is controlled by the first adjustment unit to process the text vector and initial image based on the target feature mapping rule to generate the initial foreground layer.

[0199] In some embodiments, after the foreground layer generation module extracts global text features by processing text vectors and extracts global image features by processing the initial image, the device can further be used to: determine whether local precision control is needed when generating the foreground layer based on the global text features and / or global image features using a first adjustment unit; when it is determined that local precision control is needed, determine the precision enhancement area in the foreground layer based on the global text features and / or global image features using the first adjustment unit; adjust the feature weights of the foreground layer generation module for the precision enhancement area using the first adjustment unit; and process the text vectors and the initial image using the adjusted foreground layer generation module to generate the initial foreground layer.

[0200] In some embodiments, when the device is specifically implemented, the preset poster generation model can be trained in the following manner: Constructing an initial foreground layer generation module, an initial background layer generation module, an initial layout control module, and an initial layer blending module; wherein the initial foreground layer generation module is equipped with a first adjustment unit, and the initial background layer generation module is equipped with a second adjustment unit; combining the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module; connecting the first adjustment unit and the second adjustment unit using a specified data channel to obtain an initial poster generation model; acquiring sample data that meets the requirements; and constructing multiple objective functions for the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module; and training the initial poster generation model using the sample data based on the multiple objective functions to obtain a preset poster generation model that meets the requirements.

[0201] In some embodiments, when the device is specifically implemented, it can obtain sample data that meets the requirements in the following manner: acquiring a sample poster image; splitting the sample poster image to obtain corresponding sample foreground layer and sample background layer; determining a sample layout mask for the sample poster image based on the sample poster image, sample foreground layer, and sample background layer; and combining the sample poster image, sample foreground layer, sample background layer, and sample layout mask to obtain sample data that meets the requirements.

[0202] In some embodiments, the plurality of objective functions may specifically include a first objective function, a second objective function, a third objective function, and a fourth objective function corresponding to the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer fusion module in the initial poster generation model, respectively; wherein, different objective functions have different loss constraints.

[0203] In some embodiments, when the device is specifically implemented, a first objective function corresponding to the initial foreground layer generation module can be constructed according to the following formula:

[0204]

[0205] in, The value of the first objective function. Indicates based on arrive Expectations The poster image generated at the current time point, where t is the current time point. The poster image generated at the initial time point. The foreground layer generated by the module for the current time point is the output foreground layer. For transparency loss, The opacity channel data value of the foreground layer. For the purpose of transparency, The weighting coefficients for the transparency loss term. This is the transparency channel.

[0206] In some embodiments, when the device is specifically implemented, a second objective function corresponding to the initial background layer generation module can be constructed according to the following formula:

[0207]

[0208] in, The value of the second objective function. The background layer output by the module that generates the background image for the current time point. For style matching loss, This represents the function for calculating the style matrix. The weighting coefficients for the style matching loss term. For consistency loss items, This represents the consistency feature extraction function. The weighting coefficients for the consistency loss term.

[0209] In some embodiments, when the device is specifically implemented, the initial poster generation model can be trained using the sample data based on the plurality of objective functions in the following manner: during the model training process, the first adjustment unit and the second adjustment unit dynamically adjust the first noise addition rate control parameter of the foreground layer generation module and the second noise addition rate control parameter of the background layer generation module according to the cumulative duration between the current time point and the initial time point, and / or the layer area of ​​interest at the current time point.

[0210] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0211] As can be seen from the above, based on the poster generation device provided in the embodiments of this specification, before specific implementation, an initial poster generation model containing multiple initial modules can be constructed; simultaneously, multiple objective functions for the multiple initial modules are constructed; then, based on the multiple objective functions, the initial poster generation model is trained using sample data to obtain a preset poster generation model that meets the requirements; wherein, the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and, the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected. In specific implementation, the acquired poster description text, initial image, poster layout mask, and other multimodal data can be spliced ​​according to preset splicing rules to obtain corresponding target combination data; then, the preset poster generation model is used to process the above target combination data to generate a target poster that meets the requirements. Thus, it is possible to efficiently and accurately automatically generate target posters involving complex layer structures with good effects and high quality, effectively reducing the workload of relevant technical personnel.

[0212] In a specific scenario example, the poster generation method provided in this manual can be used to achieve intelligent poster generation based on a diffusion model. The specific implementation process may include the following:

[0213] In this scenario example, to achieve accurate intelligent poster generation, the following considerations are taken: 1) Multi-layer output diffusion model design: Design a diffusion model that can output foreground and background layers. The foreground and background are controlled by their respective LoRA (Low-Rank Adaptation) models to ensure consistent lighting between the foreground and background, and harmonious content integration. 2) Layout control: Introduce a poster layout mask to control the position of the foreground on the background, ensuring that the foreground can be directly superimposed on the background to generate a seamlessly integrated poster. 3) Local resolution enhancement: When generating the foreground layer, optimize the transparency generation accuracy of cutouts and complex details by increasing the local resolution. 4) Diverse poster layouts: Support multiple layout methods, including bottom, top, left, right, center, bottom left, bottom right, top left, and top right.

[0214] Based on the above considerations, during the training phase, a diffusion model can be used to train the foreground layer (e.g., a foreground layer generation module) and the background layer (e.g., a background layer generation module); and their respective LoRA models (e.g., a first adjustment unit and a second adjustment unit) can be used for dynamic control. Furthermore, during training, by inputting image data containing a layout mask (e.g., sample data), the model learns how to generate the foreground layer at the corresponding position based on the layout mask. Specialized training is performed on the processing of cutouts and areas with complex details (e.g., detailed texture areas) to improve the accuracy of transparency generation and detail representation.

[0215] During specific training, regarding data preparation: the image containing foreground and background can be split into "foreground sub-image + background sub-image" and the corresponding position information can be labeled. Regarding the training process: for the foreground layer: input the foreground sub-image and its corresponding layout mask (marking the foreground position), allowing the diffusion model to learn the process of "initializing the image from a noisy foreground and gradually denoising to generate a clear foreground". For the background layer: input the background sub-image and its layout mask (marking the background area), and similarly allow the model to learn the process of denoising to generate the background. Regarding LoRA model control: for style control, such as making the foreground present a "hand-drawn style" and the background present a "realistic style", the LoRA model learns the feature mapping of a specific style and injects style information into the diffusion model during generation. For local precision control, for hollow / detailed areas, LoRA can strengthen the weight of local features, allowing the model to generate more refined textures in these areas.

[0216] The layout mask is a binary image (or grayscale image). Specifically, the foreground area is marked with a specific pixel value (e.g., 255) to represent "the foreground needs to be generated here"; the background area is marked with another pixel value (e.g., 0) to represent "the background needs to be generated here". For example, in the layout mask of a poster, the rectangular area where the product is located has a pixel value of 255, and the remaining background area has a pixel value of 0.

[0217] During specific training, in each training step of the diffusion model, the "original image + layout mask" is used as the input pair; input timing: input synchronously with image data (i.e., the corresponding layout mask is input at the same time when each batch of data is fed in); input method: through the input layer of the neural network, the layout mask is concatenated with the image tensor (or used as a guiding signal for the attention mechanism), so that the model learns the mapping relationship of "generating corresponding content according to the mask position".

[0218] In addition, specialized training includes: transparency accuracy training, specifically for hollowed-out areas (such as gaps in patterns), marking the "desired transparency" (e.g., Channel values), allowing the model to learn how to generate transparent effects that meet the requirements; Detail texture training: For complex textures (such as surface textures), prepare high-resolution texture samples to allow the model to learn the generation rules of fine textures.

[0219] The implementation methods for specialized training include: data augmentation, generating a large number of synthetic images containing cutouts / complex details (such as different angles and lighting) to expand the training set; loss function weighting, in the training loss, giving higher weight to the pixel error of the cutout / detail region, forcing the model to focus on optimizing these regions; multi-scale training: first training the model at low resolution to grasp the overall outline, and then focusing on detail optimization at high resolution.

[0220] During the inference phase, users can input text descriptions or initial images to generate a preliminary poster design. Then, based on the layout mask selected by the user, the foreground and background are positioned and blended. Finally, the model is used to generate a multi-layered poster output, including a foreground layer, a background layer, and the final blended poster (e.g., the target poster).

[0221] In practical applications, the input fusion method includes: descriptive text, which is converted into a vector representation through a text encoder (such as CLIP) and fused with the latent features of the diffusion model to guide the generation direction; an initial image, which serves as the "base" for generation, upon which the model modifies and fills based on the layout mask and text description (e.g., if the initial image is a blank canvas, the model generates the foreground at the corresponding position based on the mask); and model processing logic, where the neural network of the diffusion model performs multimodal fusion of text vectors, initial image features, and layout mask features, while simultaneously referring to this information during the reverse denoising process to generate the final poster design.

[0222] Specifically, different modules in the model can be designed and constructed separately.

[0223] Foreground layer generation module: Responsible for generating a foreground layer based on user-input text or images, ensuring clear layer details and appropriate transparency. This module is trained based on the following optimization objective function:

[0224]

[0225] in, The value of the first objective function. Indicates based on arrive Expectations The poster image generated at the current time point, where t is the current time point. The poster image generated at the initial time point. The foreground layer generated by the module for the current time point is the output foreground layer. For transparency loss, The opacity channel data value of the foreground layer. For the purpose of transparency, The weighting coefficients for the transparency loss term. This is the transparency channel.

[0226] The objective function described above can additionally constrain "appropriate transparency," therefore, a transparency loss term can be implicitly incorporated into the loss function (e.g., for images). (Channel regularization) ensures that the generated foreground has the expected transparency.

[0227] Background layer generation module: Generates a background layer that harmonizes with the foreground, ensuring consistent lighting and harmonious content. The objective function of this module is as follows:

[0228]

[0229] in, The value of the second objective function. The background layer output by the module that generates the background image for the current time point. For style matching loss, This represents the function for calculating the style matrix. The weighting coefficients for the style matching loss term. For consistency loss items, This represents the consistency feature extraction function. The weighting coefficients for the consistency loss term.

[0230] The objective function described above can be constrained to "coordinate with the foreground (lighting, style)". Therefore, style matching loss (such as Gram matrix loss) and lighting consistency loss are added to the loss function to unify the visual attributes of the background and the foreground.

[0231] Layout control module: Controls the relative positions of the foreground and background through a layout mask, ensuring the generated poster layout meets user requirements. The layout mask is a binary image representing the specific position of the foreground on the background.

[0232]

[0233] in, The value of the third objective function. This represents the expectation based on m, where m is the data value of the layout mask. To lay out a mask for the target, For geometric constraint functions, This is a spatial constraint loss term relating to the relative positions of the foreground and background. As the first weighting coefficient, This is the second weighting coefficient. Let i be the local component numbered i in the layout mask. For the target layout mask, select the local component numbered i. This refers to the local dynamic weight numbered i. Specifically, layout represents the layout control module, and the specific value of the aforementioned local dynamic weight can be dynamically determined based on the local complexity (e.g., the proportion of foreground in the local area, the complexity of the shape of the local area, etc.).

[0234] The objective function described above introduces a spatial constraint term: geometric constraints (distance and alignment losses) on the relative positions of the foreground and background are added to the loss function to ensure that the layout not only "matches pixels" but also conforms to visual positional rationality. Furthermore, it supports dynamic weight adjustment: the loss weights are dynamically adjusted based on the layout complexity (such as foreground proportion and shape complexity), assigning higher weights to complex layout regions to improve training accuracy.

[0235] Layer Blending Module: Seamlessly blends the foreground and background layers to generate the final poster. The blending process is optimized based on the following objective function:

[0236]

[0237] in, The value of the fourth objective function. Indicates based on arrive Expectations The foreground layer generated by the module for the current time point is the output foreground layer. The background layer output by the module that generates the background image for the current time point. For blending operators that include transparency blending and edge smoothing, For perceptual feature extraction function, To fuse the perceived loss term, These are the weighting coefficients for perceived loss. Specifically, fuse represents the layer blending module.

[0238] The objective function described above employs two main approaches. First, it features customized fusion operators: including dedicated fusion operators for transparency blending, edge smoothing, and illumination compensation, rather than simple pixel stacking, resulting in a more natural fusion outcome. Second, it utilizes multimodal loss fusion: in addition to MSE loss, it incorporates perceptual loss (such as VGG feature distance), making the fusion outcome closer to the target in terms of semantics and visual perception, rather than just pixel-level matching.

[0239] The above scenario examples validate the poster generation method provided in this manual, which supports efficient generation of multi-layer posters: it can quickly generate multi-layer posters containing foreground and background, improving design efficiency, and the multi-layer output model design makes the generated posters more flexible and creative. It supports personalized layouts: it supports multiple layout methods to meet users' personalized needs. By controlling the position of the foreground and background through layout masks, the generated poster layouts become more diverse. It has strong model detail processing capabilities: it generates high-precision transparency for hollowed-out and complex detail areas, improving poster quality. Through specialized training, it improves the generation effect of complex detail areas. It can achieve seamless integration: the foreground and background lighting are consistent, and the content is harmoniously integrated, generating high-quality posters. The model can adaptively adjust under different lighting conditions to ensure seamless integration of the foreground and background.

[0240] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0241] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0242] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0243] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0244] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0245] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A method for generating a poster, characterized in that, include: Get the poster description text, initial image, and poster layout mask; The poster description text, initial image, and poster layout mask are spliced ​​together according to preset splicing rules to obtain the corresponding target combination data; By processing the target combination data using a preset poster generation model, a target poster that meets the requirements is generated; wherein, the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and, the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

2. The method according to claim 1, characterized in that, The process of generating a target poster that meets the requirements by processing the target combination data using a preset poster generation model includes: The foreground layer generation module processes the poster description text and initial image in the target combination data to generate the initial foreground layer. The second adjustment unit is used to obtain the consistency features of the initial foreground layer through the first adjustment unit; The second adjustment unit uses the consistency feature to control the background layer generation module to process the poster description text and initial image in the target combination data, and generates an initial background layer that matches the initial foreground layer. The layout control module determines the relative layout information of the foreground layer and the background layer based on the initial foreground layer, the initial background layer, and the poster layout mask in the target combination data. The first adjustment unit controls the foreground layer generation module based on relative layout information to fine-tune the initial foreground layer, thereby obtaining the corresponding target foreground layer; the second adjustment unit controls the background generation module based on relative layout information to fine-tune the initial background layer, thereby obtaining the corresponding target background layer. The layer blending module is used to blend and stitch the target foreground layer and the target background layer according to the relative layout information to obtain a target poster that meets the requirements.

3. The method according to claim 2, characterized in that, The target foreground layer includes multiple foreground layers; wherein, each of the multiple foreground layers is assigned a corresponding priority label.

4. The method according to claim 2, characterized in that, The foreground layer generation module is also equipped with a text encoder; Accordingly, the process of using the foreground layer generation module to process the poster description text and initial image in the target combination data to generate an initial foreground layer includes: The poster description text is converted into a corresponding text vector using a text encoder. The foreground layer generation module extracts global text features by processing text vectors; the foreground layer generation module also extracts global image features by processing the initial image. The first adjustment unit determines the global style type of the poster based on the global text features and / or global image features; and determines the target feature mapping rule that matches the global style type of the poster from the preset feature mapping rule set. The first adjustment unit controls the foreground layer generation module to process the text vector and the initial image based on the target feature mapping rules to generate an initial foreground layer.

5. The method according to claim 4, characterized in that, After using the foreground layer generation module to process text vectors and extract global text features, and using the foreground layer generation module to process the initial image and extract global image features, the method further includes: The first adjustment unit uses the global text features and / or global image features to determine whether local precision control is needed when generating the foreground layer; When it is determined that local precision control is required, the first adjustment unit determines the precision enhancement area in the foreground layer based on the global text features and / or global image features; The first adjustment unit is used to adjust the feature weights of the foreground layer generation module for the precision enhancement area; The adjusted foreground layer generation module is used to process the text vector and the initial image to generate an initial foreground layer.

6. The method according to claim 1, characterized in that, The preset poster generation model is trained in the following manner: Construct an initial foreground layer generation module, an initial background layer generation module, an initial layout control module, and an initial layer blending module; wherein, the initial foreground layer generation module deploys a first adjustment unit, and the initial background layer generation module deploys a second adjustment unit; The initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module are combined; and the first adjustment unit and the second adjustment unit are connected using a specified data channel to obtain the initial poster generation model. Obtain sample data that meets the requirements; and construct multiple objective functions for the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer blending module; Based on the multiple objective functions, the initial poster generation model is trained using the sample data to obtain a preset poster generation model that meets the requirements.

7. The method according to claim 6, characterized in that, The acquisition of qualified sample data includes: Obtain sample poster images; Based on the sample poster image, the corresponding sample foreground layer and sample background layer are obtained by splitting it; Based on the sample poster image, sample foreground layer, and sample background layer, a sample layout mask for the sample poster image is determined. By combining the sample poster image, sample foreground layer, sample background layer, and sample layout mask, sample data that meets the requirements is obtained.

8. The method according to claim 6, characterized in that, The multiple objective functions include a first objective function, a second objective function, a third objective function, and a fourth objective function corresponding to the initial foreground layer generation module, the initial background layer generation module, the initial layout control module, and the initial layer fusion module in the initial poster generation model, respectively; wherein, different objective functions have different loss constraints.

9. The method according to claim 8, characterized in that, Construct the first objective function corresponding to the initial foreground layer generation module according to the following formula: in, The value of the first objective function. Indicates based on arrive Expectations The poster image generated at the current time point, where t is the current time point. The poster image generated at the initial time point. The foreground layer generated by the module for the current time point is the output foreground layer. For transparency loss, The opacity channel data value of the foreground layer. For the purpose of transparency, The weighting coefficients for the transparency loss term. This is the transparency channel.

10. The method according to claim 8, characterized in that, Construct the second objective function corresponding to the initial background layer generation module according to the following formula: in, The value of the second objective function. The background layer output by the module that generates the background image for the current time point. For style matching loss, This represents the function for calculating the style matrix. The weighting coefficients for the style matching loss term. For consistency loss items, This represents the consistency feature extraction function. The weighting coefficients for the consistency loss term.

11. The method according to claim 6, characterized in that, The step of training the initial poster generation model using the sample data based on the multiple objective functions includes: During model training, the first adjustment unit and the second adjustment unit are used to dynamically adjust the first noise addition rate control parameter of the foreground layer generation module and the second noise addition rate control parameter of the background layer generation module based on the cumulative duration between the current time point and the initial time point, and / or the layer area of ​​interest at the current time point.

12. A poster generating device, characterized in that, include: The acquisition module is used to acquire the poster description text, initial image, and poster layout mask; The splicing module is used to splice the poster description text, initial image, and poster layout mask according to preset splicing rules to obtain the corresponding target combination data; The generation module is used to generate a target poster that meets the requirements by processing the target combination data using a preset poster generation model; wherein the preset poster generation model includes at least: a foreground layer generation module, a background layer generation module, a layout control module, and a layer blending module; and the foreground layer generation module is equipped with a first adjustment unit, the background layer generation module is equipped with a second adjustment unit, and the first adjustment unit and the second adjustment unit are connected.

13. A server, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images