Poster generation method and device based on multi-stage progressive form, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a poster generation method, device, equipment and medium based on a multi-stage progressive mode, and the method comprises the steps: receiving original design elements, carrying out the analysis and feature extraction of the original design elements, and obtaining feature representations of different levels; inputting the different levels of feature representations into a pre-trained progressive generative network, and performing staged progressive layout generation processing on the different levels of feature representations to obtain corresponding adaptive layout parameters; and rendering the corresponding original design elements according to the adaptive layout parameters, and generating and displaying a corresponding poster image. Staged progressive layout generation processing is carried out on feature representations of different levels of elements, so that layout scheme generation of different levels can be focused at different stages, and the stability and layout generation quality under a multi-element poster layout scene are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for generating posters based on a multi-stage progressive method. Background Art

[0002] As an important form of visual communication, poster design's layout generation needs to meet both geometric constraints (such as element alignment and spacing control) and aesthetic requirements (such as visual balance and style consistency). Therefore, it also puts higher demands on the layout optimization of poster design.

[0003] For example, in the financial sector, when financial institutions promote new financial products, credit card services or investment strategies, they need to design attractive promotional posters. When laying out the posters, they need to consider factors such as the characteristics of the financial products and the preferences of the target customer groups, and ensure that the overall layout is in line with the professionalism and seriousness of the financial industry. For example, in the medical and health field, medical research institutions or hospitals need to use posters to display the latest research results, such as clinical trial results, medical paper abstracts, research methods, etc., so when laying out the posters, they need to consider factors such as the importance and relevance of the research content, so as to more effectively convey the research content.

[0004] Traditional poster layout design relies primarily on designer experience or rule-based algorithms, making it difficult to adapt to diverse design needs. In recent years, deep learning technologies have been introduced into the field of layout generation. For example, generative adversarial networks (GANs) have been introduced to layout generation tasks, and graph-constrained GANs have been proposed for generating architectural layouts. However, this approach is prone to mode collapse or gradient conflicts due to the global game between the generator and the discriminator, making it difficult to ensure layout quality and stability in complex design scenarios such as multi-element poster layouts. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a multi-stage progressive poster generation method, device, equipment and medium that can be applied to the medical field, financial technology or other related fields. Its main purpose is to improve the stability and layout generation quality in the multi-element poster layout scenario.

[0006] The technical solutions of the present invention are as follows:

[0007] A first aspect of the present invention provides a poster generation method based on a multi-stage progressive approach, comprising:

[0008] Receiving original design elements, parsing and extracting features from the original design elements, and obtaining feature representations at different levels;

[0009] Inputting the feature representations at different levels into a pre-trained progressive generation network, performing phased progressive layout generation processing on the feature representations at different levels to obtain corresponding adaptive layout parameters;

[0010] The corresponding original design elements are rendered according to the adaptive layout parameters to generate and display a corresponding poster image.

[0011] A second aspect of the present invention provides a poster generation device based on a multi-stage progressive method, comprising:

[0012] An element parsing module is used to receive original design elements, parse and extract features from the original design elements, and obtain feature representations at different levels;

[0013] A progressive layout generation module, configured to input the feature representations at different levels into a pre-trained progressive generation network, perform progressive layout generation processing on the feature representations at different levels in stages, and obtain corresponding adaptive layout parameters;

[0014] The rendering and display module is used to render the corresponding original design elements according to the adaptive layout parameters, and generate and display the corresponding poster image.

[0015] A third aspect of the present invention provides a computer device comprising at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned poster generation method based on multi-stage progressive approach.

[0018] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned multi-stage progressive poster generation method.

[0019] Beneficial effects: The present invention discloses a poster generation method, apparatus, device and medium based on multi-stage progressiveness. Compared with the prior art, the embodiment of the present invention receives original design elements, parses and extracts features from the original design elements, and obtains feature representations at different levels; inputs the feature representations at different levels into a pre-trained progressive generation network, performs progressive layout generation processing on the feature representations at different levels in stages, and obtains corresponding adaptive layout parameters; renders the corresponding original design elements according to the adaptive layout parameters, and generates and displays the corresponding poster image. By performing progressive layout generation processing on the feature representations at different levels of the elements in stages, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the solutions in the present invention, a brief introduction is given below to the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A schematic diagram of an application environment for the multi-stage progressive poster generation method provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of a multi-stage progressive poster generation method provided by an embodiment of the present invention;

[0023] Figure 3 A flowchart of step S201 in the multi-stage progressive poster generation method provided in an embodiment of the present invention;

[0024] Figure 4 A flowchart of step S202 in the multi-stage progressive poster generation method provided in an embodiment of the present invention;

[0025] Figure 5 A flowchart of the progressive generation network training phase in the multi-stage progressive poster generation method provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of the functional modules of a multi-stage progressive poster generation device provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and effects of the present invention more clear and distinct, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below with reference to the accompanying drawings.

[0029] The poster generation method based on multi-stage progressiveness provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, the system includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0031] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0032] The server 105 may be a server that provides various services, such as a backend server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The backend server may analyze and process the received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0033] It should be noted that the poster generation method based on the multi-stage progressive method provided in the embodiments of the present application can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the poster generation apparatus based on the multi-stage progressive method provided in the embodiments of the present invention can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the poster generation method based on the multi-stage progressive method provided in the embodiments of the present invention can generally be executed by the server 105. Accordingly, the poster generation apparatus based on the multi-stage progressive method provided in the embodiments of the present invention can generally be set in the server 105.

[0034] It should be understood that the numbers of the above terminal devices, networks and servers are merely illustrative and any number of terminal devices, networks and servers may be provided as required.

[0035] like Figure 2 As shown, the poster generation method based on multi-stage progressiveness provided by the embodiment of the present invention specifically includes the following steps:

[0036] S201: Receive original design elements, parse and extract features from the original design elements, and obtain feature representations at different levels.

[0037] In this embodiment, the original design elements required for poster design uploaded by the user, that is, the unprocessed basic materials, are received. The original design elements can be text (such as titles, subtitles, and text), images (such as product images, background images, etc.), and other possible graphic elements (such as icons, lines, etc.). Preferably, for poster designs that need to be dynamically updated, it can also support the acquisition of design elements from real-time data streams (such as stock quotes and medical monitoring data) to improve the real-time nature of poster design. By quickly extracting the basic elements required for design from multiple data sources, data support is provided for subsequent processing.

[0038] The received raw design elements are parsed to obtain attribute data for each element. For example, text content is parsed into strings, images are parsed into pixel matrices, and metadata (such as file format and size) is extracted. Based on the element parsing results, a feature extraction network (such as a multi-scale feature extractor based on ConvNeXt) is used to extract features from the parsed elements to obtain feature representations at different levels for each raw design element. For example, the feature representations include low-level texture and edge information, as well as high-level semantic information. These feature representations can reflect the core information of the design elements from different levels, and can simultaneously capture local details and global information, providing accurate and rich input feature representations for multi-stage targeted layout generation processing.

[0039] For example, in the financial field, when designing a financial advertising poster, the input original design elements may include the logo of the financial institution, interest rate information text, product pictures, etc. Through parsing and feature extraction, the feature representations of different levels of these elements are extracted to provide support for subsequent layout generation.

[0040] In the healthcare field, when designing medical posters, the input raw design elements may include images of medical equipment, health tips, hospital logos, etc. These elements are parsed and corresponding features are extracted at different resolutions or scales to obtain feature representations at different levels.

[0041] S202: Input the feature representations at different levels into a pre-trained progressive generation network, perform progressive layout generation processing on the feature representations at different levels in stages, and obtain corresponding adaptive layout parameters.

[0042] In this embodiment, a progressive generation network is pre-built and trained. The progressive generation network can adopt a hierarchical Transformer-U-Net architecture to generate high-quality layouts by gradually refining layout parameters, thereby realizing a progressive stage-by-stage generation of layouts from coarse to fine. Specifically, after the extracted feature representations of different levels are input into the progressive generation network, the layout generation task is decomposed into multiple stages. In each stage, the layout characteristics of different levels are focused on. The layout generated in the previous stage is further refined and outputted as the layout scheme of the current stage, and a stage-by-stage progressive layout generation process is performed to generate the final adaptive layout parameters, including the position, size, spacing and other attributes of the original design elements in the poster, so as to adapt to different poster design tasks. In this embodiment, for multi-element design scenarios, the feature representations of different levels of elements can be subjected to stage-by-stage progressive layout generation processing, which can avoid the problem of mode collapse or gradient conflict easily caused by global game and improve the stability of layout generation.

[0043] For example, when designing promotional posters for financial products in the financial field, feature representations at different levels are input into a progressive generative network for phased progressive layout generation. For example, layout plans are generated in phases based on the input financial product information (such as interest rates, risk warnings, etc.) and brand image elements (such as logos, color schemes, etc.). First, the position of the brand logo is determined, then the layout of the text content is optimized, and finally the overall visual balance is adjusted.

[0044] When designing medical and health posters in the medical and health field, feature representations at different levels are input into the progressive generative network for phased progressive layout generation. For example, layout plans are generated in stages based on the input medical information (such as disease prevention knowledge, hospital contact information, etc.) and visual elements (such as medical equipment images, health icons, etc.). First, the position of the medical equipment image is determined, then the image spacing is adjusted, the text area size is optimized, and finally the visual balance is fine-tuned to ensure aesthetic quality.

[0045] S203: Render the corresponding original design elements according to the adaptive layout parameters to generate and display a corresponding poster image.

[0046] In this embodiment, based on the adaptive layout parameters obtained through the staged progressive layout generation process, the layout coordinates, size, rotation angle and other parameters of each original design element are obtained, and then the original design elements are rendered according to the optimized layout parameters through a rendering engine, real-time rendering, etc., and the original design elements are placed in the corresponding positions, and visual adjustments are made (such as font size, color, image cropping, etc.), so as to convert the layout parameters into a visual image, and the rendered poster image is displayed to the user in a visual manner for easy viewing and feedback by the user.

[0047] For example, in the design of a financial advertising poster, elements such as the financial institution's logo, product information text, and interest rate table are rendered on the poster according to the generated adaptive layout parameters. For example, the logo is placed at the top, the product information text is placed in the middle, and the interest rate table is placed at the bottom, generating a poster that conforms to the financial brand image.

[0048] In the design of medical promotional posters, elements such as medical equipment images, health reminder texts, and hospital logos are rendered onto the posters according to adaptive layout parameters. For example, the medical equipment images are placed in a prominent position, the health reminder texts are placed below, and the hospital logos are placed in the corners, generating posters that meet the requirements of medical promotion.

[0049] In the above embodiment, the present invention discloses a poster generation method based on multi-stage progressiveness, which receives original design elements, parses and extracts features from the original design elements to obtain feature representations at different levels; inputs the feature representations at different levels into a pre-trained progressive generation network, performs progressive layout generation processing on the feature representations at different levels in stages to obtain corresponding adaptive layout parameters; renders the corresponding original design elements according to the adaptive layout parameters, and generates and displays the corresponding poster image. By performing progressive layout generation processing on the feature representations at different levels of the elements in different stages, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios.

[0050] In one embodiment, Figure 3 As shown, step S201 includes:

[0051] S301, receiving original design elements, and parsing the original design elements into structured data;

[0052] S302 : Extracting multi-scale features at different levels from the structured data using a multi-scale feature extractor to generate feature representations at different levels.

[0053] In this embodiment, original design elements provided by the user are received through a user interface or API, and the original design elements are parsed into structured data. For text elements, the text elements can be converted into a string format through a text parser, and the style information of the text (such as font, font size, color, etc.) can be extracted; for image elements, they can be converted into a pixel matrix through an image parser, and the metadata of the image (such as size, format, resolution, etc.) can be extracted; for vector graphic elements, the acquirer path information and attributes (such as color, fill, line style, etc.) can be parsed, and the parsed element attribute related information can be converted into a structured data format, such as JSON or XML, thereby converting the unstructured input data into a format with a clear data structure, providing a basis for feature extraction.

[0054] Afterwards, a pre-trained multi-scale feature extractor, such as a ConvNeXt-based feature extraction network, is loaded and the parsed structured data is input into the multi-scale feature extractor. A multi-scale feature extractor is a tool or model that can extract features at different resolutions or scales from input data. For text data, semantic features and style features (such as font size and color) are extracted; for image data, low-level texture and edge information, as well as high-level semantic information, are extracted; and for vector graphics data, shape features (such as path length and curvature) and style features (such as color and fill) are extracted. The extracted multi-scale features are constructed into feature representations at different levels, such as low-level features (texture and edge information), mid-level features (local shape and color distribution), and high-level features (semantic information and overall layout information). Multi-scale feature extraction can simultaneously capture local details and global information of the input elements, integrating features at different levels into a complete feature representation, providing rich input for subsequent layout generation.

[0055] In one embodiment, Figure 4 As shown, step S202 includes:

[0056] S401, inputting the feature representations of different levels into a pre-trained progressive generative network, wherein the progressive generative network includes a plurality of sequentially connected stage generators;

[0057] S402: In the first stage generator, layout prediction is performed based on the feature representation of the level corresponding to the first stage to generate the layout features of the first stage;

[0058] S403: In the second-stage generator, layout prediction is performed based on the layout features of the first stage and the feature representation of the corresponding level of the second stage to generate the layout features of the second stage;

[0059] S404, and so on, in each stage generator, the layout features of the current stage are generated according to the layout features output by the previous stage generator and the feature representation of the corresponding level of the current stage, until the layout features output by the last stage generator are obtained;

[0060] S405 , performing feature conversion based on the layout features output by the final stage generator to generate corresponding adaptive layout parameters.

[0061] In this embodiment, the progressive generation network includes multiple stage generators connected in sequence, and the multiple stage generators are obtained through stage-by-stage adversarial learning training. In each stage generator, different levels of layout feature prediction can be focused on. After the feature representations of different levels are input into the progressive generation network, in the first stage generator, the layout prediction is performed based on the feature representation of the level corresponding to the first stage to generate the layout features of the first stage, that is, the feature level corresponding to the first stage is selected from the input multi-scale feature representation. For example, the first stage generator is responsible for coarse-grained element positioning, and it may focus on the coarse-grained element position and the overall layout structure, generate preliminary layout features, including the approximate position of the elements, the overall framework of the layout, etc., and pass the layout features generated in the first stage to the next stage generator.

[0062] In the second-stage generator, the layout features generated in the first stage are fused with the corresponding feature representations in the second stage. The feature representations in the second stage may focus on more fine-grained layout information, such as the relative position and spacing between elements. Based on these fused features, the second-stage layout prediction generates even more refined layout features. For example, the second-stage generator is responsible for optimizing the basic relationships between elements. The generated layout features include the spacing between elements and the order of their arrangement. This allows for refinement based on the initial layout and optimization of the relationships between elements. This gradual optimization avoids the difficulties of processing complex layouts all at once and improves generation quality.

[0063] Repeat the above-mentioned second-stage process and pass through each stage generator in turn. Each stage generator optimizes the layout based on the layout features of the previous stage and the feature representation of the current stage. For example, the third-stage generator is responsible for processing fine alignment and spacing, and the last-stage generator is responsible for the final aesthetic adjustment, etc., so that each stage generator further optimizes the layout based on the previous stage, gradually refining the layout features from coarse-grained to fine-grained, and outputs the final layout features through the last-stage generator. The layout features output by the last-stage generator define the final state of the layout. Through multi-stage optimization, a high-quality layout is generated to meet aesthetic and functional requirements.

[0064] Through feature conversion methods such as linear mapping and nonlinear mapping, the layout features output by the final stage generator are converted into specific adaptive layout parameters. For example, the position information in the feature representation is converted into specific coordinate values, and the size information is converted into specific width and height values, etc., thereby converting the abstract layout features into specific layout parameters to facilitate subsequent rendering and display.

[0065] In one embodiment, before step S203, the method further includes:

[0066] The adaptive layout parameters are constrained and optimized according to preset poster design criteria to generate optimized adaptive layout parameters.

[0067] In this embodiment, a set of poster design criteria is preset to constrain optimization of adaptive layout parameters. Specific poster design criteria include spacing between elements, alignment, hierarchy, aesthetic standards, etc. For example, text elements should be located at the top of the poster, image elements should occupy the center, and icons should be evenly distributed around the edges. The adaptive layout parameters are further optimized based on the design criteria. For example, an optimization objective function is constructed based on each design criterion, and the layout parameters are adjusted by minimizing the objective function using an optimization algorithm (such as gradient descent). This optimizes the adaptive layout parameters so that the optimized adaptive layout parameters meet the poster design criteria and have high aesthetic quality, thereby achieving further layout optimization.

[0068] In one embodiment, after step S203, the method further includes:

[0069] Performing layout evaluation on the poster image and / or receiving layout feedback on the poster image from a user to obtain layout evaluation information;

[0070] The adaptive layout parameters are fine-tuned accordingly according to the layout evaluation information, and a corresponding poster image is re-rendered based on the fine-tuned layout solution.

[0071] In this embodiment, after generating a visual poster image, the generated poster image is automatically evaluated using pre-set layout evaluation indicators. For example, the overall aesthetics of the poster, such as balance, symmetry, and visual appeal, are evaluated; the rationality of the distribution of elements, such as whether they are evenly distributed or whether there is overlap, is evaluated; and the readability of text elements, such as font size and color contrast, are evaluated. Automatic evaluation indicators are used to quickly obtain a preliminary evaluation of the layout, improving efficiency. A user interface is also provided to receive user feedback on the layout of the generated poster image. User feedback is used to collect subjective opinions and ensure that the generated poster meets user needs. At least one of the automatic evaluation results and user feedback is used as layout evaluation information to meet different layout evaluation requirements.

[0072] Based on the layout evaluation information, layout parameters that need adjustment are extracted. For example, if user feedback indicates that the title is too small, the title font size is adjusted; if the automatic evaluation finds overlapping elements, the element positions are adjusted, and so on. Based on the layout evaluation information, the adaptive layout parameters are fine-tuned accordingly, and the poster image is re-rendered using the fine-tuned layout parameters. The new poster image is displayed to the user, who is asked to make further adjustments or confirmations. This layout fine-tuning based on evaluation information further optimizes the layout quality, ensures that the generated poster meets user needs, and improves user satisfaction with the generated poster.

[0073] In one embodiment, Figure 5 As shown in Figure 2, the progressive generative network is trained through multi-stage adversarial learning by the following steps:

[0074] S501, constructing a progressive generative network including multiple stage generators and an adversarial network including multiple local adversarial critics;

[0075] S502: Collect training design elements and corresponding real layout samples, parse and extract features of the training design elements, and generate corresponding training feature representations;

[0076] S503, performing layout prediction on the training feature representation in stages by the multiple stage generators to obtain a predicted layout output by each stage generator;

[0077] S504. Using a local adversarial critic corresponding to each stage generator, perform a stage-by-stage layout evaluation on the predicted layout output by each stage generator according to the real layout sample to obtain the adversarial loss and perceptual loss of each stage.

[0078] S505. Dynamically calculate the total training loss of each stage based on the adversarial loss and perceptual loss of each stage, and adjust the parameters of the progressive generative network and the adversarial network according to the total training loss until the preset conditions are met, thereby completing the multi-stage adversarial learning training.

[0079] In this embodiment, in the training phase of the progressive generative network, it is specifically obtained through multi-stage adversarial learning training, thereby avoiding mode collapse or gradient conflict caused by the global generative adversarial game. First, a progressive generative network is constructed, which includes multiple stage generators. Each stage generator is responsible for processing layout optimization tasks at a specific level, such as the first stage generator is responsible for coarse-grained element positioning, the second stage generator is responsible for optimizing the basic relationship between elements, and the third stage generator is responsible for processing fine alignment and spacing; and a corresponding adversarial network is constructed, which includes multiple local adversarial critics, each critic corresponding to a stage generator, used to evaluate the layout quality output by the stage generator, such as the first local adversarial critic is responsible for evaluating the rationality of the basic position of the elements, the second local adversarial critic is responsible for checking the spatial relationship between elements, and the third local adversarial critic is responsible for verifying alignment and spacing. By constructing staged generators and critics, layout generation and evaluation at different levels are performed in stages to solve the gradient conflict problem of the global discriminator.

[0080] Collect a large number of training design elements and their corresponding real layout samples. These training design elements are collected from existing design works or manually annotated data sets, etc. The collected design elements are parsed and feature extracted to generate corresponding training feature representations. The specific parsing and extraction process is similar to the inference stage and will not be described here. The training feature representation is input into the progressive generative network. The multiple stage generators in the generative network perform staged layout prediction on the training feature representation to obtain the predicted layout output by the generator at each stage. Specifically, the progressive generative network adopts a hierarchical Transformer-U-Net architecture, which includes s generation stages. The generation process of each stage s can be expressed as:

[0081] L s =G s (L s-1 ,F s )

[0082] Among them L s-1 is the layout feature generated in the previous stage, F s is the multi-scale feature of the current stage, extracted by the feature extraction network, G s is the generator of the sth stage. That is, in each stage generator, the layout features of the current stage are generated according to the layout features output by the previous stage generator and the feature representation of the corresponding level of the current stage, until the layout features output by the last stage generator are obtained.

[0083] Correspondingly, the predicted layout and the true layout samples output by the generator at each stage are input into the corresponding local adversarial critic. The local adversarial critic evaluates the authenticity of the predicted layout at each stage by comparing the predicted and true layouts and calculating the adversarial loss. It also calculates the perceptual difference between the predicted and true layouts using a perceptual loss function (such as the perceptual loss based on CLIP-ViT). This staged evaluation avoids the gradient conflict problem of the global discriminator, and the adversarial loss and perceptual loss provide multi-dimensional feedback to help the generator optimize the layout.

[0084] Based on the dynamics of the adversarial and perceptual losses at each stage, an exponential decay strategy is then used to dynamically adjust the weights of the adversarial and perceptual losses. This allows the total training loss to be dynamically calculated for each stage. This ensures the appropriate focus at each training stage, such as ensuring structural correctness in the early stages and aesthetic quality in the later stages. The backpropagation algorithm adjusts the parameters of the progressive generative network and adversarial network based on the total training loss. Training is terminated when the training loss converges or reaches the preset number of training rounds, completing the multi-stage adversarial learning training.

[0085] In this embodiment, the progressive generative network adopts a multi-stage adversarial learning training method, based on phased optimization and dynamic adjustment of loss weights, and gradually refines the layout from coarse-grained to fine-grained. At each layout generation stage, it is evaluated by the corresponding local adversarial critic to solve the gradient conflict problem of the global discriminator, effectively improving the quality and stability of layout generation.

[0086] In one embodiment, the total training loss at each stage is dynamically calculated based on the adversarial loss and perceptual loss at each stage, including:

[0087] Calculate the loss weights at each stage based on the preset initial adversarial loss weights and attenuation coefficients;

[0088] According to the loss weight of each stage, the adversarial loss and perceptual loss of the corresponding stage are weighted summed to calculate the total training loss of each stage.

[0089] In this embodiment, when dynamically calculating the total training loss of each stage, the weight coefficients of different losses are dynamically adjusted according to the preset initial adversarial loss weight and the decay coefficient through the exponential decay strategy. The preset initial adversarial loss weight determines the importance of the adversarial loss in the early stage of training, and the decay coefficient is used to adjust the weight change of the adversarial loss in the subsequent stages. For each stage s, the formula λ is used. s =λ0·γ s To calculate the adversarial loss weight of the current stage, where λ0 is the initial adversarial loss weight, γ is the decay coefficient (0<γ<1), and λ s is the adversarial loss weight of the sth stage, based on λ sThe perceptual loss weight of the sth stage can be obtained as 1-λ s The adversarial loss weight is dynamically adjusted through the attenuation coefficient to ensure that the initial training phase focuses on structural correctness and the later phase focuses on aesthetic quality.

[0090] For each stage s, obtain the adversarial loss of the generator at that stage and perceptual loss The adversarial loss and perceptual loss of the corresponding stage are weighted summed according to the loss weight of each stage:

[0091]

[0092] Then calculate the total training loss R at each stage s , by taking into account the adversarial loss and perceptual loss through weighted summation, we can ensure that the generator is optimized in both structure and aesthetics. This scheduling strategy can ensure that in the early stage, i.e., λ s When it is large, it focuses on structural correctness, and in the later stage, λ s When smaller, it focuses on aesthetic quality and smooth transitions to avoid training instability.

[0093] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0094] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a poster generation device based on a multi-stage progressive method. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0095] like Figure 6 As shown, the poster generation device 60 based on multi-stage progressive method described in this embodiment includes:

[0096] An element parsing module 601 is configured to receive original design elements, parse and extract features from the original design elements, and obtain feature representations at different levels;

[0097] A progressive layout generation module 602 is configured to input the feature representations at different levels into a pre-trained progressive generation network, perform progressive layout generation processing on the feature representations at different levels in stages, and obtain corresponding adaptive layout parameters;

[0098] The rendering and display module 603 is used to render the corresponding original design elements according to the adaptive layout parameters, and generate and display the corresponding poster image.

[0099] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable for describing the poster generation execution process based on a multi-stage progressive approach than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0100] In one embodiment, the element parsing module 601 includes:

[0101] A parsing unit, configured to receive original design elements and parse the original design elements into structured data;

[0102] The feature extraction unit is used to extract multi-scale features of different levels in the structured data through a multi-scale feature extractor to generate feature representations of different levels.

[0103] In one embodiment, the progressive layout generation module 602 includes:

[0104] A feature input unit, configured to input the feature representations of different levels into a pre-trained progressive generative network, wherein the progressive generative network comprises a plurality of sequentially connected stage generators;

[0105] A progressive generation control unit is used to perform layout prediction in the first-stage generator based on the feature representation of the level corresponding to the first stage, and generate the layout features of the first stage;

[0106] In the second stage generator, layout prediction is performed based on the layout features of the first stage and the feature representation of the corresponding level of the second stage to generate the layout features of the second stage;

[0107] Similarly, in each stage generator, the layout features of the current stage are generated according to the layout features output by the previous stage generator and the feature representation of the corresponding level of the current stage, until the layout features output by the last stage generator are obtained and converted;

[0108] The conversion unit is used to perform feature conversion based on the layout features output by the final stage generator and generate corresponding adaptive layout parameters.

[0109] In one embodiment, the apparatus 60 further includes:

[0110] The constraint optimization module is used to perform constraint optimization on the adaptive layout parameters according to preset poster design criteria to generate optimized adaptive layout parameters.

[0111] In one embodiment, the apparatus 60 further includes:

[0112] Building blocks for constructing progressive generative networks with multiple stage generators and adversarial networks with multiple local adversarial critics;

[0113] An acquisition and extraction module is used to acquire training design elements and corresponding real layout samples, parse and extract features of the training design elements, and generate corresponding training feature representations;

[0114] A staged generation module, configured to perform staged layout prediction on the training feature representation through the multiple stage generators, and obtain a predicted layout output by each stage generator;

[0115] A local adversarial critic module is used to perform a stage-by-stage layout evaluation on the predicted layout output by each stage generator according to the real layout samples through a local adversarial critic corresponding to each stage generator, and obtain the adversarial loss and perceptual loss of each stage;

[0116] The loss calculation and parameter optimization module is used to dynamically calculate the total training loss of each stage based on the adversarial loss and perceptual loss of each stage, and adjust the parameters of the progressive generative network and the adversarial network according to the total training loss until the preset conditions are met to complete the multi-stage adversarial learning training.

[0117] In one embodiment, the loss calculation and parameter optimization module includes:

[0118] The weight calculation unit is used to calculate the loss weight of each stage according to the preset initial adversarial loss weight and attenuation coefficient;

[0119] The loss calculation unit is used to perform weighted summation of the adversarial loss and perceptual loss of the corresponding stage according to the loss weight of each stage to calculate the total training loss of each stage.

[0120] In one embodiment, the apparatus 60 further includes:

[0121] an evaluation collection module, configured to perform layout evaluation on the poster image and / or receive user feedback on the layout of the poster image to obtain layout evaluation information;

[0122] A feedback fine-tuning module is used to perform corresponding fine-tuning processing on the adaptive layout parameters according to the layout evaluation information, and re-render and generate a corresponding poster image based on the fine-tuned layout parameters.

[0123] In the above embodiment, the present invention discloses a poster generation device based on multi-stage progressiveness, which receives original design elements, parses and extracts features from the original design elements to obtain feature representations at different levels; inputs the feature representations at different levels into a pre-trained progressive generation network, performs progressive layout generation processing on the feature representations at different levels in stages to obtain corresponding adaptive layout parameters; renders the corresponding original design elements according to the adaptive layout parameters, and generates and displays the corresponding poster image. By performing progressive layout generation processing on the feature representations at different levels of the elements in stages, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios.

[0124] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0125] One or more processors 701 and memory 702, Figure 7 In the description, a processor 701 is used as an example. The processor 701 and the memory 702 can be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0126] The processor 701 is used to complete various control logics of the computer device 70. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination of these components. In addition, the processor 701 can also be any traditional processor, microprocessor or state machine. The processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0127] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as program instructions corresponding to the multi-stage progressive poster generation method in the embodiments of the present invention. Processor 701 executes the non-volatile software programs, instructions, and modules stored in memory 702 to execute various functional applications and data processing of computer device 70, thereby implementing the multi-stage progressive poster generation method in the above-mentioned method embodiment.

[0128] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely located relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the multi-stage progressive poster generation method in any of the above-mentioned method embodiments are executed.

[0129] In the above-mentioned embodiment, the present invention discloses a computer device that receives original design elements, parses and extracts features from the original design elements to obtain feature representations at different levels; inputs the feature representations at different levels into a pre-trained progressive generation network, performs phased progressive layout generation processing on the feature representations at different levels to obtain corresponding adaptive layout parameters; renders the corresponding original design elements according to the adaptive layout parameters, and generates and displays a corresponding poster image. By performing phased progressive layout generation processing on the feature representations at different levels of the elements, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios.

[0130] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the multi-stage progressive poster generation method in any of the above method embodiments are executed.

[0131] In the above-mentioned embodiment, the present invention discloses a non-volatile computer-readable storage medium, which receives original design elements, parses and extracts features from the original design elements to obtain feature representations at different levels; inputs the feature representations at different levels into a pre-trained progressive generation network, performs phased progressive layout generation processing on the feature representations at different levels, and obtains corresponding adaptive layout parameters; renders the corresponding original design elements according to the adaptive layout parameters, and generates and displays the corresponding poster image. By performing phased progressive layout generation processing on the feature representations at different levels of the elements, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0133] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0134] In summary, the present invention discloses a multi-stage progressive poster generation method, apparatus, device, and medium, the method comprising: receiving original design elements, parsing and extracting features from the original design elements to obtain feature representations at different levels; inputting the feature representations at different levels into a pre-trained progressive generation network, performing a phased progressive layout generation process on the feature representations at different levels to obtain corresponding adaptive layout parameters; rendering the corresponding original design elements according to the adaptive layout parameters to generate and display the corresponding poster image. By performing a phased progressive layout generation process on the feature representations at different levels of the elements, it is possible to focus on generating layout solutions at different levels at different stages, thereby improving the stability and layout generation quality in multi-element poster layout scenarios.

[0135] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. The storage medium can be a memory, a magnetic disk, a floppy disk, a flash memory, an optical storage device, etc.

[0136] It should be noted that if any software tools or components not developed by our company appear in the examples of this application, they are for illustration purposes only and do not represent actual use. It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the scope of protection of the appended claims.

Claims

1. A poster generation method based on multi-stage progressive method, characterized in that: include: Receiving original design elements, parsing and extracting features from the original design elements, and obtaining feature representations at different levels; Inputting the feature representations at different levels into a pre-trained progressive generation network, performing phased progressive layout generation processing on the feature representations at different levels to obtain corresponding adaptive layout parameters; The corresponding original design elements are rendered according to the adaptive layout parameters to generate and display a corresponding poster image.

2. The poster generation method based on multi-stage progressiveness according to claim 1 is characterized in that: The receiving of original design elements, parsing and feature extraction of the original design elements, and obtaining feature representations at different levels include: Receiving original design elements, and parsing the original design elements into structured data; The multi-scale features at different levels in the structured data are extracted by a multi-scale feature extractor to generate feature representations at different levels.

3. The poster generation method based on multi-stage progressive method according to claim 1, characterized in that: The step of inputting the feature representations at different levels into a pre-trained progressive generation network, performing phased progressive layout generation processing on the feature representations at different levels, and obtaining corresponding adaptive layout parameters includes: Inputting the feature representations at different levels into a pre-trained progressive generative network, wherein the progressive generative network includes a plurality of sequentially connected stage generators; In the first stage generator, layout prediction is performed based on the feature representation of the corresponding level of the first stage to generate the layout features of the first stage; In the second stage generator, layout prediction is performed based on the layout features of the first stage and the feature representation of the corresponding level of the second stage to generate the layout features of the second stage; Similarly, in each stage generator, the layout features of the current stage are generated according to the layout features output by the previous stage generator and the feature representation of the corresponding level of the current stage, until the layout features output by the last stage generator are obtained; Feature conversion is performed based on the layout features output by the generator in the final stage to generate the corresponding adaptive layout parameters.

4. The poster generation method based on multi-stage progressiveness according to claim 1 is characterized in that: Before rendering the corresponding original design elements according to the adaptive layout parameters and generating and displaying the corresponding poster image, the method further includes: The adaptive layout parameters are constrained and optimized according to preset poster design criteria to generate optimized adaptive layout parameters.

5. The poster generation method based on multi-stage progressive method according to claim 1 is characterized in that: The progressive generative network is obtained by multi-stage adversarial learning training through the following steps: Build a progressive generative network consisting of multiple stage generators and an adversarial network consisting of multiple local adversarial critics; Collecting training design elements and corresponding real layout samples, parsing and extracting features from the training design elements, and generating corresponding training feature representations; Performing stage-by-stage layout prediction on the training feature representation by the multiple stage generators to obtain a predicted layout output by each stage generator; Through the local adversarial critic corresponding to each stage generator, the predicted layout output by each stage generator is evaluated in stages according to the real layout samples to obtain the adversarial loss and perceptual loss of each stage; The total training loss of each stage is dynamically calculated based on the adversarial loss and perceptual loss of each stage, and the parameters of the progressive generative network and the adversarial network are adjusted according to the total training loss until the preset conditions are met, completing the multi-stage adversarial learning training.

6. The poster generation method based on multi-stage progressiveness according to claim 5, characterized in that: The total training loss of each stage is dynamically calculated based on the adversarial loss and perceptual loss of each stage, including: Calculate the loss weights at each stage based on the preset initial adversarial loss weights and attenuation coefficients; According to the loss weight of each stage, the adversarial loss and perceptual loss of the corresponding stage are weighted summed to calculate the total training loss of each stage.

7. The poster generation method based on multi-stage progressiveness according to any one of claims 1 to 6, characterized in that: After rendering the corresponding original design elements according to the adaptive layout parameters and generating and displaying the corresponding poster image, the method further includes: Performing layout evaluation on the poster image and / or receiving layout feedback on the poster image from a user to obtain layout evaluation information; The adaptive layout parameters are fine-tuned accordingly according to the layout evaluation information, and a corresponding poster image is re-rendered based on the fine-tuned layout parameters.

8. A poster generation device based on multi-stage progressive method, characterized in that: include: An element parsing module is used to receive original design elements, parse and extract features from the original design elements, and obtain feature representations at different levels; A progressive layout generation module, configured to input the feature representations at different levels into a pre-trained progressive generation network, perform progressive layout generation processing on the feature representations at different levels in stages, and obtain corresponding adaptive layout parameters; The rendering and display module is used to render the corresponding original design elements according to the adaptive layout parameters, and generate and display the corresponding poster image.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the poster generation method based on multi-stage progressiveness according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the multi-stage progressive poster generation method according to any one of claims 1 to 7.

Images