Poster generation method and device based on sparse gradient pulse, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a poster generation method, device and equipment based on sparse gradient pulses and a medium, and the method comprises the steps: obtaining original input elements, carrying out the analysis and feature extraction of the original input elements, and generating corresponding element feature representations; the element feature representation is input into a pre-trained layout generator for layout scheme prediction, corresponding adaptive layout parameters are generated, and the layout generator is obtained based on sparse gradient pulse mechanism training; and rendering the original input elements according to the adaptive layout parameters, and generating and displaying a corresponding poster image. The layout generator obtained through training based on the sparse gradient pulse mechanism carries out layout prediction processing on the input elements, and a parameter structure obtained through training after sparsification is utilized to realize more efficient and faster prediction reasoning speed, so that the memory occupation of the model is reduced, and the poster layout generation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a poster generation method, device, equipment and medium based on sparse gradient pulses. Background Art

[0002] Poster element layout generation is a key issue at the intersection of computer-aided design and generative artificial intelligence. Its goal is to automatically generate a two-dimensional spatial arrangement scheme that meets aesthetic and functional requirements given given visual elements (such as images and text).

[0003] For example, in the financial sector, financial institutions often need to design promotional posters to promote financial products, such as wealth management products, insurance products, and credit card promotions. These posters need to display key information (such as product name, interest rate, promotional content, risk warnings, etc.) in a limited space, while attracting customers' attention and conveying a professional and reliable image. Another example is in the medical and health field, hospitals, clinics, public health institutions, etc. need to design promotional posters to promote medical services, health knowledge, and public welfare activities. Posters need to clearly display key information (such as medical project introductions, expert consultation information, health lecture times, etc.) and convey a sense of professionalism and trust to attract the attention of patients and the public.

[0004] Because traditional poster design methods rely on heuristic rules or optimization-based layout algorithms, they struggle to handle complex design constraints and diverse style requirements. In recent years, deep learning technologies, such as Transformer-based layout generation models and generative adversarial networks (GANs), have significantly improved layout quality through end-to-end training. However, this approach suffers from high computational complexity and low efficiency, hindering the efficiency of poster layout generation. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a poster generation method, device, equipment and medium based on sparse gradient pulses that can be applied to the medical field, financial technology or other related fields. Its main purpose is to save the computational cost of poster layout tasks and improve the efficiency of poster layout generation.

[0006] The technical solutions of the present invention are as follows:

[0007] A first aspect of the present invention provides a poster generation method based on sparse gradient pulses, comprising:

[0008] Obtaining original input elements, parsing and extracting features from the original input elements, and generating corresponding element feature representations;

[0009] Inputting the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism;

[0010] The original input element is rendered according to the adaptive layout parameters to generate and display a corresponding poster image.

[0011] A second aspect of the present invention provides a poster generation device based on sparse gradient pulses, comprising:

[0012] A parsing and extraction module is used to obtain original input elements, parse and extract features from the original input elements, and generate corresponding element feature representations;

[0013] a layout generation module, configured to input the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism;

[0014] The rendering and display module is used to render the original input elements according to the adaptive layout parameters, and generate and display a corresponding poster image.

[0015] A third aspect of the present invention provides a computer device comprising at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above poster generation method based on sparse gradient pulses.

[0018] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned poster generation method based on sparse gradient pulses.

[0019] Beneficial effects: The present invention discloses a poster generation method, apparatus, device and medium based on sparse gradient pulses. Compared with the prior art, the embodiments of the present invention obtain the original input elements, parse and extract features of the original input elements, and generate corresponding element feature representations; input the element feature representations into a pre-trained layout generator to predict the layout scheme and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient pulse mechanism; render the original input elements according to the adaptive layout parameters to generate and display the corresponding poster image. The layout generator trained based on the sparse gradient pulse mechanism performs layout prediction processing on the input elements, and the parameter structure obtained after sparse training is used to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the solutions in the present invention, a brief introduction is given below to the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A schematic diagram of an application environment of the poster generation method based on sparse gradient pulses provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of a poster generation method based on sparse gradient pulses provided by an embodiment of the present invention;

[0023] Figure 3 A flowchart of the layout generator training phase in the poster generation method based on sparse gradient impulses provided by an embodiment of the present invention;

[0024] Figure 4 A flowchart of step S304 in the poster generation method based on sparse gradient pulses provided in an embodiment of the present invention;

[0025] Figure 5 A flowchart of step S401 in the poster generation method based on sparse gradient pulses provided in an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of the functional modules of a poster generation device based on sparse gradient pulses provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and effects of the present invention more clear and distinct, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below with reference to the accompanying drawings.

[0029] The poster generation method based on sparse gradient pulses provided by the embodiment of the present invention can be applied in the following fields: Figure 1 In an application environment, the system includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0031] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0032] The server 105 may be a server that provides various services, such as a backend server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The backend server may analyze and process the received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0033] It should be noted that the poster generation method based on sparse gradient pulses provided in the embodiments of the present application can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the poster generation apparatus based on sparse gradient pulses provided in the embodiments of the present invention can also be provided in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the poster generation method based on sparse gradient pulses provided in the embodiments of the present invention can generally be executed by the server 105. Accordingly, the poster generation apparatus based on sparse gradient pulses provided in the embodiments of the present invention can generally be provided in the server 105.

[0034] It should be understood that the numbers of the above terminal devices, networks and servers are merely illustrative and any number of terminal devices, networks and servers may be provided as required.

[0035] like Figure 2 As shown, the poster generation method based on sparse gradient pulses provided by the embodiment of the present invention specifically includes the following steps:

[0036] S201: Acquire original input elements, parse and extract features from the original input elements, and generate corresponding element feature representations.

[0037] In this embodiment, when designing a poster, users can input the original input elements of the layout to be designed by uploading files through the interface or retrieving existing materials from the database. The specific original input elements can be in various forms such as text and images. For example, in the poster design scenario, text elements may include titles, subtitles, and body text; image elements may include product images and background images.

[0038] The obtained raw input elements are parsed and features are extracted. For example, the parsing of text elements includes text format recognition (such as font size, style, etc.) and text content extraction; the parsing of image elements includes image size detection, image format recognition, graphic resolution detection and content extraction. Further feature extraction is performed based on the parsing results, such as extracting the semantic features of text elements and the visual features of image elements, and then splicing them together to obtain the overall element feature representation. Alternatively, the text and image elements can be directly encoded into a high-dimensional feature representation through a visual language model. Through element parsing and feature extraction, the raw input elements are converted into feature representations that the model can understand and process, ensuring that the semantic and visual information of the input elements are fully extracted and utilized, thereby improving the quality of layout generation.

[0039] For example, in the design of financial advertising posters, the input elements may include the logo image of the financial institution, product introduction text (such as financial product yields, risk warnings, etc.), and background images (such as chart images of financial markets). Through parsing and feature extraction, the corresponding element feature representation is generated, so that the model can understand the brand image, product characteristics and visual style of the financial institution.

[0040] In medical and health poster design, input elements may include images of medical equipment, health promotion texts (such as disease prevention knowledge, healthy lifestyle recommendations, etc.), and background images (such as hospital environment images). Through parsing and feature extraction, corresponding element feature representations are generated to provide support for subsequent layout generation.

[0041] S202: Input the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism.

[0042] In this embodiment, the element feature representation obtained through feature extraction is used as input to the layout generator. Since the element feature representation contains the semantic and visual information of text and image elements, it provides comprehensive input data for the layout generator. Specifically, the layout generator is based on the Transformer-decoder architecture and consists of an encoder and a decoder. The encoder encodes the input feature sequence into a context representation, and the decoder generates an output sequence based on the context representation. In the layout generation task, the decoder is responsible for generating layout parameters based on the input element feature representation, that is, generating a layout plan by predicting the coordinate position of each element in two-dimensional space.

[0043] The pre-trained layout generator can calculate the optimal position of each element in the poster, including coordinates, size and other parameters, based on the layout optimization pattern learned during the training phase and the input element feature representation, and then output adaptive layout parameters to achieve adaptive generation processing of the poster layout. Among them, the layout generator is trained using a sparse gradient pulse mechanism. The sparse gradient pulse mechanism dynamically sparsifies the gradient and schedules parameter updates during the training phase, retaining only significant gradient components for parameter updates. This allows for greater emphasis on training parameters that have a significant impact on layout quality, while reducing other unnecessary calculations. This allows the layout generator trained after sparsification to achieve more efficient and faster prediction and inference speeds during the inference phase, reducing the model's memory usage and improving the efficiency of poster layout generation.

[0044] For example, in the design of financial advertising posters, the feature representations of elements such as the financial institution logo image, product introduction text and background image are input into the layout generator for layout prediction to generate adaptive layout parameters. For example, the financial institution logo is placed in the middle of the poster, the product introduction text is arranged on the left in a clear hierarchical structure, and the background image is used as a backdrop to highlight the characteristics and advantages of the financial product.

[0045] In medical health poster design, the feature representations of elements such as medical device images, health promotion text, and background images are input into the layout generator for layout prediction, generating a corresponding adaptive layout solution. For example, the medical device image is placed in the center of the poster, and the health promotion text is arranged around the image in a concise and clear manner. The background image serves as a supporting element to create a professional and comfortable medical atmosphere, effectively conveying health information and raising public health awareness.

[0046] S203: Render the original input element according to the adaptive layout parameters to generate and display a corresponding poster image.

[0047] In this embodiment, based on the adaptive layout parameters output by the layout generator, the layout coordinates, size, rotation angle, and other parameters of each original design element are obtained. The original design elements are rendered according to the adaptive layout parameters through a rendering engine, real-time rendering, etc., and the original design elements are placed in the corresponding positions. Visual adjustments (such as font size, color, image cropping, etc.) are then made, thereby converting the layout parameters into a visual image. The rendered poster image is then visually displayed to the user. The generated layout scheme is presented in a visual manner, allowing the user to intuitively see the final poster effect, facilitating user evaluation and adjustment, thereby improving the quality of the poster design.

[0048] For example, in the design of a financial advertising poster, elements such as the financial institution's logo, product introduction text, and interest rate table are rendered on the poster according to the generated adaptive layout parameters. For example, the logo is placed at the top, the product introduction text is placed in the middle, and the interest rate table is placed at the bottom, generating a poster that conforms to the financial brand image and attracts the attention of potential customers.

[0049] In medical and health poster design, elements such as medical device images, health promotion texts, and hospital logos are rendered according to adaptive layout parameters. For example, the medical device image is placed in a prominent position, the health promotion text is placed below, and the hospital logo is placed in the corner to generate a poster for health promotion activities.

[0050] In the above embodiment, the present invention discloses a poster generation method based on sparse gradient pulses, which obtains original input elements, parses and extracts features from the original input elements, and generates corresponding element feature representations; inputs the element feature representations into a pre-trained layout generator to predict the layout scheme and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient pulse mechanism; renders the original input elements according to the adaptive layout parameters to generate and display the corresponding poster image. The layout generator trained based on the sparse gradient pulse mechanism performs layout prediction processing on the input elements, and the parameter structure obtained after sparsification training is used to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation.

[0051] In one embodiment, step S201 includes:

[0052] Obtaining an original input element, and parsing the original input element into structured data;

[0053] The multimodal model is used to uniformly encode the features of the data of different modalities in the structured data to generate corresponding element feature representations.

[0054] In this embodiment, the user can obtain the original input element by uploading text, image and other element files through the interactive interface, or by retrieving the corresponding element data from the local storage. The obtained original input element is parsed into structured data. For text elements, the text element can be converted into a string format through a text parser, and the style information of the text (such as font, font size, color, etc.) can be extracted; for image elements, it can be converted into a pixel matrix through an image parser, and the metadata of the image (such as size, format, resolution, etc.) can be extracted. The parsed element attribute related information is converted into a structured data format, such as JSON or XML, thereby converting the unstructured input data into a format with a clear data structure, providing a basis for feature extraction.

[0055] Then, a pre-trained multimodal model, such as the Visual Language Transformer (ViLT), is loaded as a feature extractor. Multimodal models are deep learning models capable of simultaneously processing data from multiple modalities (such as text and images). For example, the ViLT model encodes and fuses text and images through a shared Transformer architecture. The multimodal model uniformly encodes features from different modalities within the structured data, capturing both the semantic features of the text and the visual features of the image to generate corresponding element feature representations. The feature vector of each element contains both semantic and visual information for subsequent layout generation, resulting in a layout that better meets semantic and visual requirements.

[0056] In one embodiment, after step S203, the method further includes:

[0057] Performing layout evaluation on the poster image and / or receiving layout feedback on the poster image from a user to obtain layout evaluation information;

[0058] The adaptive layout parameters are fine-tuned accordingly according to the layout evaluation information, and a corresponding poster image is re-rendered based on the fine-tuned layout parameters.

[0059] In this embodiment, after generating a visual poster image, the generated poster image is automatically evaluated using pre-set layout evaluation indicators. For example, the overall aesthetics of the poster, such as balance, symmetry, and visual appeal, are evaluated; the rationality of the distribution of elements, such as whether they are evenly distributed or whether there is overlap, is evaluated; and the readability of text elements, such as font size and color contrast, are evaluated. Automatic evaluation indicators are used to quickly obtain a preliminary evaluation of the layout, improving efficiency. A user interface is also provided to receive user feedback on the layout of the generated poster image. User feedback is used to collect subjective opinions and ensure that the generated poster meets user needs. At least one of the automatic evaluation results and user feedback is used as layout evaluation information to meet different layout evaluation requirements.

[0060] Based on the layout evaluation information, layout parameters that need adjustment are extracted. For example, if user feedback indicates that the title is too small, the title font size is adjusted; if the automatic evaluation finds overlapping elements, the element positions are adjusted, and so on. Based on the layout evaluation information, the adaptive layout parameters are fine-tuned accordingly, and the poster image is re-rendered using the fine-tuned layout parameters. The new poster image is displayed to the user, who is asked to make further adjustments or confirmations. This layout fine-tuning based on evaluation information further optimizes the layout quality, ensures that the generated poster meets user needs, and improves user satisfaction with the generated poster.

[0061] In one embodiment, Figure 3 As shown in Figure 2, the layout generator is trained based on the sparse gradient impulse mechanism through the following steps:

[0062] S301, collecting training design elements and corresponding real layout parameters, parsing and extracting features from the training design elements, and generating corresponding training feature representations;

[0063] S302: Input the training feature representation into the layout generator in the current training step to perform layout scheme prediction and obtain predicted layout parameters;

[0064] S303, calculating an original gradient value of a loss function according to a difference between the predicted layout parameters and the actual layout parameters;

[0065] S304, performing dynamic sparse processing on the original gradient value under the control of a dynamic threshold by a gradient pulse controller to obtain a sparse gradient value;

[0066] S305, updating the parameters of the layout generator in the current training step according to the sparse gradient value and then proceeding to the next training step;

[0067] S306: cyclically execute the above process in each training step until a trained layout generator is obtained when a preset condition is met.

[0068] In this embodiment, during the training phase of the layout generator, gradient sparse processing is performed under the dynamic control of the gradient pulse controller to improve training efficiency and reduce memory usage. Specifically, a large number of poster design samples are first collected, including training design elements such as text and images and their corresponding real layout parameters (such as the position and size of the elements), to ensure the diversity and representativeness of the training data, covering different design styles, element types and layout requirements. The collected training design elements are parsed, and the text and image elements are converted into structured data. The structured data is feature extracted through a multimodal model (such as ViLT) to generate a high-dimensional feature representation for each design element. The training feature representation is input as training data into the layout generator for layout optimization training, which provides high-quality input data and supervision signals for the training process, ensuring that the model can learn an effective layout generation strategy. Specifically, given a set of input elements where each x i It can be text or image elements, and feature extraction is performed through the ViLT model:

[0069]

[0070] Among them, h i is the d-dimensional feature representation of the i-th element, n is the total number of elements, and the feature representations of all elements are concatenated into the design context matrix H = [h1,…,h n ] Input into the layout generator to generate the layout.

[0071] Specifically, the training feature representation is input into the layout generator, and the layout generator f θ It is a Transformer-decoder with parameter θ, which can predict the parameters of each design element in the poster based on the input feature representation Based on the predicted layout parameters Y and the actual layout parameters Y * The difference between the loss function is calculated And the original gradient value of the loss function The specific loss function can measure the difference between the predicted layout parameters and the actual layout parameters through mean square error, cross entropy loss, etc., and the gradient value calculated by back propagation represents the derivative of the loss function with respect to the model parameters, providing a basis for subsequent parameter updates.

[0072] When updating parameters based on gradient values, unlike the standard neural network training process, this embodiment first uses a gradient pulse controller to dynamically thin out the original gradient values ​​under the control of a dynamic threshold, obtaining sparse gradient values ​​for subsequent parameter updates. Specifically, the gradient pulse controller analyzes factors such as gradient amplitude and layout score feedback to generate a dynamic threshold. This dynamic threshold is then used to thin out the original gradient values, setting the portion of the original gradient value below the threshold to zero. Only significant gradient components are retained for subsequent parameter updates. This thinned-out gradient value allows the model parameters that have the greatest impact on layout quality to be identified and prioritized for updating, reducing unnecessary computation and memory usage.

[0073] Based on the generated sparse gradient values, the model parameters of the layout generator are updated in the current training step to optimize the performance of the layout generator. After completing the parameter update of the current training step, the next training step is entered and the above process is repeated. That is, the above steps of predicting layout, calculating original gradients, gradient sparsification, and parameter adjustment are repeated in each training step, gradually optimizing the performance of the layout generator until the preset conditions are met, such as reaching the maximum number of training steps, the loss function converges to less than the preset threshold, etc., then the training is stopped to obtain the trained layout generator. Through the training process based on the sparse gradient pulse mechanism, the layout generator can reduce unnecessary computation and memory overhead during training, give priority to updating the parameters of significant gradient components, and more efficiently learn effective layout generation strategies to provide support for subsequent reasoning tasks.

[0074] In one embodiment, Figure 4 As shown, step S304 includes:

[0075] S401, obtaining the target sparsity rate of the current training step;

[0076] S402, performing exponential moving average processing according to the target sparsity rate and the dynamic threshold of the previous training step to obtain the dynamic threshold of the current training step;

[0077] S403 , generating a corresponding binary mask according to the dynamic threshold of the current training step, and performing a sparse processing by multiplying the binary mask by the original gradient value element by element to obtain a corresponding sparse gradient value.

[0078] In this embodiment, when dynamically sparsifying the original gradient values, the target sparsity rate for the current training step is first obtained. The target sparsity rate determines the proportion of gradient components to be retained in the current training step. For example, a target sparsity rate of 0.3 indicates that 30% of the gradient components are desired to be retained. The target sparsity rate can be fixed or dynamically adjusted. The specific sparsity rate strategy can be set based on task requirements and model characteristics. For example, the sparsity rate can be gradually reduced as the number of training steps increases, allowing for rapid exploration of the parameter space in the early stages of training and fine-tuning in the later stages.

[0079] The dynamic threshold of the current training step is obtained by performing exponential moving average processing based on the target sparsity rate of the current training step and the dynamic threshold of the previous training step. That is, the dynamic threshold of the current training step is related to the target sparsity rate and the historical dynamic threshold, and is dynamically adjusted through exponential moving average:

[0080] τ t =ατ t-1 +(1-α)·Quantile(|g t |,p)

[0081] Among them, τ t is the dynamic threshold of the current training step t, τ t-1 is the dynamic threshold of the previous training step t-1, α is the smoothing coefficient (such as 0.9), p is the target sparsity rate (such as 0.3), and the Quantile function returns the p quantile of the gradient amplitude. t is the original gradient value at training step t. Dynamic threshold adjustment is performed based on gradient amplitude exponential smoothing and quantile statistics, so that the dynamic threshold can smoothly combine historical and current information, avoid drastic fluctuations in the threshold, and ensure the adaptability of the sparsification process.

[0082] According to the dynamic threshold τ of the current training step t , generate the corresponding binary mask m t , specifically, That is, for each gradient component g t , if its absolute value is greater than or equal to the dynamic threshold, the mask value is 1, otherwise it is 0, thereby indicating which gradient components need to be retained (value 1) and which need to be set to zero (value 0). The generated binary mask is multiplied element by element with the original gradient value to perform sparse processing and obtain the corresponding sparse gradient value, that is, Where ⊙ represents element-by-element multiplication, and the sparse gradient value Participate in subsequent parameter updates. By multiplying the binary mask with the original gradient value element by element, unimportant gradient components are set to zero. The sparse gradient value reduces unnecessary calculations and only retains significant gradient components, thereby significantly improving training efficiency.

[0083] In one embodiment, Figure 5 As shown, step S401 includes:

[0084] S501, performing a perceptual layout scoring on the predicted layout parameters to obtain a corresponding layout scoring value;

[0085] S502 : Adaptively adjust the basic sparsity rate according to the layout score value and a preset score target value to obtain a target sparsity rate for the current training step.

[0086] In this embodiment, the target sparsity rate of the current training step is obtained based on an adaptive feedback mechanism, and the target sparsity rate is adaptively adjusted through perceptual layout scoring (PLS) feedback. Specifically, the perceptual layout scoring is performed based on the predicted layout parameters (such as the position and size of the elements), that is, the quality of the layout is evaluated. The layout scoring system can be based on pre-designed scoring criteria such as aesthetic rules (such as balance, symmetry, contrast, etc.) and functional requirements (such as the clarity of information communication). The predicted layout parameters are input into the perceptual layout scoring system, and the layout is evaluated according to the preset scoring criteria to output a layout scoring value. The layout scoring value reflects the quality of the current predicted layout. The higher the scoring value, the more the layout meets the aesthetic and functional requirements, thereby performing a quantitative evaluation of the predicted layout and providing a basis for subsequent sparsity rate adjustment.

[0087] The basic sparsity rate is adaptively adjusted based on the layout score and the preset score target value. Specifically, before training begins, a layout score target value is set according to the task requirements. This target value represents the expected layout quality level. The sparsity rate adjustment is calculated based on the difference between the current layout score and the preset score target value. For example, if the current score is lower than the target value, the sparsity rate is increased to explore more parameter updates; if the current score is close to the target value, the sparsity rate is reduced to fine-tune the adjustment. The adjustment is specifically made using the following formula:

[0088] p←p+λ·ReLU(PLS current -PLS target )

[0089] Among them, λ is the adjustment step size, ReLU is the activation function, and PLS current is the current layout score, PLS target It is a preset score target value, ensuring that the sparsity rate is increased only when the current score is lower than the target. By dynamically adjusting the sparsity rate, the model can adaptively balance the range between exploration and convergence during training, achieving reduced computational overhead while maintaining layout generation performance.

[0090] In one embodiment, before step S305, the method further includes:

[0091] Dividing the parameters of the layout generator into several parameter groups in advance for periodic cyclic activation;

[0092] Step S305 specifically includes:

[0093] The sparse gradient value is input into a preset optimizer, and the optimizer updates the currently activated parameter group according to the sparse gradient value and then enters the next training step.

[0094] In this embodiment, when performing parameter updates, a cyclic scheduler is used to manage the alternating activation and training of parameter groups, thereby implementing a cyclic parameter update mechanism. Specifically, before training begins, all parameters of the layout generator are divided into several parameter groups. For example, the parameters can be grouped according to their position or function in the model, or randomly grouped. A periodic activation strategy is also predefined to determine which parameter groups are activated in each training step. For example, a rotating activation method is used, activating one parameter group per training step, or activating multiple parameter groups in sequence in a certain order, etc., and different parameter groups are periodically activated according to a preset strategy to perform cyclic parameter updates.

[0095] When performing cyclic parameter updates, only the currently active parameter group is updated. For example, the parameters are divided into C groups. In period k, only group θ k Receive updates:

[0096]

[0097] Where η is the learning rate, is an indicator function that is 1 if the parameter θ belongs to group θk and 0 otherwise.

[0098] When the AdamW optimizer is integrated for parameter update, after the sparse gradient value is obtained in the current training step, it is input into the preset optimizer, for example, the improved AdamW optimizer is input to perform momentum correction on the sparse gradient value. The optimizer updates the currently activated parameter group according to the sparse gradient value and then enters the next training step:

[0099]

[0100] Where β2 is the momentum coefficient, ε is the numerical stability term, and v t is the second-order moment estimate of training step t, v t-1 The second-order moment estimate at training step t-1, It is the deviation correction of the second-order moment. By integrating the AdamW optimizer to perform momentum correction on the sparse gradient, the learning rate can be dynamically adjusted to improve the stability and convergence speed of training. By periodically activating different parameter groups, the optimizer can balance the update speed of different parameter groups, avoiding the problem of some parameter groups updating too quickly while other parameter groups update too slowly, thereby improving the training efficiency and performance of the model.

[0101] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0102] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a poster generation device based on sparse gradient pulses, which is similar to the embodiment of the present invention. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0103] like Figure 6 As shown, the poster generation device 60 based on sparse gradient pulses described in this embodiment includes:

[0104] The parsing and extraction module 601 is used to obtain the original input element, parse and extract features from the original input element, and generate corresponding element feature representations;

[0105] a layout generation module 602 configured to input the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism;

[0106] The rendering and display module 603 is used to render the original input element according to the adaptive layout parameters, and generate and display a corresponding poster image.

[0107] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable for describing the execution process of poster generation based on sparse gradient pulses than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0108] In one embodiment, the parsing and extraction module 601 includes:

[0109] A parsing unit, configured to obtain original input elements and parse the original input elements into structured data;

[0110] The feature extraction unit is used to perform unified feature encoding on data of different modalities in the structured data through a multimodal model to generate corresponding element feature representations.

[0111] In one embodiment, the apparatus 60 further includes:

[0112] The acquisition and extraction module is used to acquire training design elements and corresponding real layout parameters, analyze and extract features of the training design elements, and generate corresponding training feature representations.

[0113] An input prediction module is used to input the training feature representation into the layout generator in the current training step to predict the layout scheme and obtain predicted layout parameters;

[0114] a gradient calculation module, configured to calculate an original gradient value of a loss function according to a difference between the predicted layout parameters and the actual layout parameters;

[0115] a sparse processing module, configured to perform dynamic sparse processing on the original gradient value under the control of a dynamic threshold through a gradient pulse controller to obtain a sparse gradient value;

[0116] A parameter updating module, configured to update the parameters of the layout generator in the current training step according to the sparse gradient value and then proceed to the next training step;

[0117] The loop control module is used to loop through the above process in each training step until a trained layout generator is obtained when a preset condition is met.

[0118] In one embodiment, the sparse processing module includes:

[0119] The sparsity rate acquisition unit is used to obtain the target sparsity rate of the current training step;

[0120] A threshold adjustment unit, configured to perform exponential moving average processing based on the target sparsity rate and the dynamic threshold of the previous training step to obtain the dynamic threshold of the current training step;

[0121] The sparse unit is used to generate a corresponding binary mask according to the dynamic threshold of the current training step, and multiply the binary mask by the original gradient value element by element to perform a sparse processing to obtain a corresponding sparse gradient value.

[0122] In one embodiment, the sparse rate acquisition unit includes:

[0123] a layout scoring unit, configured to perform a perceptual layout scoring on the predicted layout parameters to obtain a corresponding layout scoring value;

[0124] The sparsity rate adaptive unit is used to adaptively adjust the basic sparsity rate according to the layout score value and a preset score target value to obtain the target sparsity rate of the current training step.

[0125] In one embodiment, the device 60 comprises:

[0126] A cyclic activation module, used to pre-divide the parameters of the layout generator into several parameter groups for periodic cyclic activation;

[0127] The parameter updating module is specifically used to:

[0128] The sparse gradient value is input into a preset optimizer, and the optimizer updates the currently activated parameter group according to the sparse gradient value.

[0129] In one embodiment, the apparatus 60 further includes:

[0130] an evaluation module, configured to perform layout evaluation on the poster image and / or receive user feedback on the layout of the poster image to obtain layout evaluation information;

[0131] A feedback fine-tuning module is used to perform corresponding fine-tuning processing on the adaptive layout parameters according to the layout evaluation information, and re-render and generate a corresponding poster image based on the fine-tuned layout parameters.

[0132] In the above embodiment, the present invention discloses a poster generation device based on sparse gradient pulses, which obtains original input elements, parses and extracts features from the original input elements, and generates corresponding element feature representations; inputs the element feature representations into a pre-trained layout generator to predict layout schemes and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient pulse mechanism; renders the original input elements according to the adaptive layout parameters, and generates and displays corresponding poster images. The layout generator trained based on the sparse gradient pulse mechanism performs layout prediction processing on the input elements, and utilizes the parameter structure obtained after sparse training to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation.

[0133] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0134] One or more processors 701 and memory 702, Figure 7 In the description, a processor 701 is used as an example. The processor 701 and the memory 702 can be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0135] The processor 701 is used to complete various control logics of the computer device 70. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination of these components. In addition, the processor 701 can also be any traditional processor, microprocessor or state machine. The processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0136] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions corresponding to the poster generation method based on sparse gradient pulses in the embodiments of the present invention. Processor 701 executes the non-volatile software programs, instructions, and modules stored in memory 702 to execute various functional applications and data processing of computer device 70, thereby implementing the poster generation method based on sparse gradient pulses in the above-mentioned method embodiments.

[0137] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely located relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the poster generation method based on sparse gradient pulses in any of the above-mentioned method embodiments are executed.

[0138] In the above embodiment, the present invention discloses a computer device that obtains original input elements, parses and extracts features from the original input elements, and generates corresponding element feature representations; inputs the element feature representations into a pre-trained layout generator to predict layout schemes and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism; renders the original input elements according to the adaptive layout parameters to generate and display corresponding poster images. The layout generator trained based on the sparse gradient impulse mechanism performs layout prediction processing on the input elements, and utilizes the parameter structure obtained after sparsification training to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation.

[0139] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the poster generation method based on sparse gradient pulses in any of the above method embodiments are performed.

[0140] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which obtains original input elements, parses and extracts features from the original input elements, and generates corresponding element feature representations; inputs the element feature representations into a pre-trained layout generator to predict layout schemes and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism; and renders the original input elements according to the adaptive layout parameters to generate and display corresponding poster images. The layout generator trained based on the sparse gradient impulse mechanism performs layout prediction processing on the input elements, and utilizes the parameter structure obtained after sparsification training to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation.

[0141] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0142] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0143] In summary, the poster generation method, apparatus, device and medium based on sparse gradient pulse disclosed in the present invention include: obtaining original input elements, parsing and extracting features from the original input elements, and generating corresponding element feature representations; inputting the element feature representations into a pre-trained layout generator to predict the layout scheme and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on the sparse gradient pulse mechanism; rendering the original input elements according to the adaptive layout parameters to generate and display the corresponding poster image. The layout generator trained based on the sparse gradient pulse mechanism performs layout prediction processing on the input elements, and the parameter structure obtained after sparse training is used to achieve a more efficient and faster prediction and inference speed, thereby reducing the memory usage of the model and improving the efficiency of poster layout generation.

[0144] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. The storage medium can be a memory, a magnetic disk, a floppy disk, a flash memory, an optical storage device, etc.

[0145] It should be noted that if any software tools or components not developed by our company appear in the examples of this application, they are for illustration purposes only and do not represent actual use. It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the scope of protection of the appended claims.

Claims

1. A poster generation method based on sparse gradient pulses, characterized in that: include: Obtaining original input elements, parsing and extracting features from the original input elements, and generating corresponding element feature representations; Inputting the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism; The original input element is rendered according to the adaptive layout parameters to generate and display a corresponding poster image.

2. The poster generation method based on sparse gradient pulse according to claim 1, characterized in that: The obtaining of the original input element, parsing and feature extraction of the original input element, and generating a corresponding element feature representation includes: Obtaining an original input element, and parsing the original input element into structured data; The multimodal model is used to uniformly encode the features of the data of different modalities in the structured data to generate corresponding element feature representations.

3. The poster generation method based on sparse gradient pulse according to claim 1, characterized in that: The layout generator is trained based on the sparse gradient impulse mechanism through the following steps: Collecting training design elements and corresponding real layout parameters, parsing and extracting features from the training design elements, and generating corresponding training feature representations; Inputting the training feature representation into the layout generator in the current training step to predict the layout scheme and obtain predicted layout parameters; Calculating an original gradient value of a loss function according to a difference between the predicted layout parameters and the true layout parameters; Performing dynamic sparse processing on the original gradient value under the control of a dynamic threshold by a gradient pulse controller to obtain a sparse gradient value; After updating the parameters of the layout generator in the current training step according to the sparse gradient value, proceed to the next training step; The above process is executed in a loop at each training step until a trained layout generator is obtained when the preset conditions are met.

4. The poster generation method based on sparse gradient pulse according to claim 3, characterized in that: The step of dynamically performing a sparse processing on the original gradient value under the control of a dynamic threshold by a gradient pulse controller to obtain a sparse gradient value includes: Get the target sparsity rate of the current training step; Performing exponential moving average processing according to the target sparsity rate and the dynamic threshold of the previous training step to obtain the dynamic threshold of the current training step; A corresponding binary mask is generated according to the dynamic threshold of the current training step, and the binary mask is multiplied element-by-element by the original gradient value to perform a sparse processing to obtain a corresponding sparse gradient value.

5. The poster generation method based on sparse gradient pulse according to claim 4, characterized in that: The obtaining of the target sparsity rate of the current training step includes: Performing a perceptual layout score on the predicted layout parameters to obtain a corresponding layout score value; The basic sparsity rate is adaptively adjusted according to the layout score value and a preset score target value to obtain the target sparsity rate of the current training step.

6. The poster generation method based on sparse gradient pulse according to claim 3, characterized in that: After updating the parameters of the layout generator in the current training step according to the sparse gradient value and before entering the next training step, the method further includes: Dividing the parameters of the layout generator into several parameter groups in advance for periodic cyclic activation; The step of updating the parameters of the layout generator according to the sparse gradient value in the current training step and then entering the next training step specifically includes: The sparse gradient value is input into a preset optimizer, and the optimizer updates the currently activated parameter group according to the sparse gradient value and then enters the next training step.

7. The poster generation method based on sparse gradient pulse according to any one of claims 1 to 6, characterized in that: After rendering the original input element according to the adaptive layout parameters and generating and displaying a corresponding poster image, the method further includes: Performing layout evaluation on the poster image and / or receiving layout feedback on the poster image from a user to obtain layout evaluation information; The adaptive layout parameters are fine-tuned accordingly according to the layout evaluation information, and a corresponding poster image is re-rendered based on the fine-tuned layout parameters.

8. A poster generation device based on sparse gradient pulses, characterized in that: include: A parsing and extraction module is used to obtain original input elements, parse and extract features from the original input elements, and generate corresponding element feature representations; a layout generation module, configured to input the element feature representation into a pre-trained layout generator to predict a layout solution and generate corresponding adaptive layout parameters, wherein the layout generator is trained based on a sparse gradient impulse mechanism; The rendering and display module is used to render the original input elements according to the adaptive layout parameters, and generate and display a corresponding poster image.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the poster generation method based on sparse gradient pulses according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the poster generation method based on sparse gradient pulses according to any one of claims 1 to 7.

Images