Poster generation method and device based on dynamic neural architecture, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a poster generation method, device and equipment based on a dynamic neural architecture and a medium, and the method comprises the following steps: obtaining original design elements, and carrying out analysis and feature extraction to obtain corresponding element feature vectors; inputting the element feature vectors into a pre-constructed layout generator, and dynamically adjusting a neural network structure of the layout generator according to the feature vectors to obtain a layout generator of a dynamic architecture; carrying out coding processing on the inter-element relationship on the element feature vector through a layout generator of a dynamic architecture, and outputting layout relationship features; performing optimization conversion according to the layout relation characteristics to generate corresponding optimization layout parameters; and rendering the original design elements according to the optimized layout parameters, and generating and displaying a corresponding poster image. The neural network structure of the layout generator is dynamically adjusted based on different design tasks, so that the calculation consumption is reduced, and the calculation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a poster generation method, device, equipment and medium based on a dynamic neural architecture. Background Art

[0002] Poster design is an important form of visual communication. The rationality of its element layout directly affects the efficiency of information transmission and aesthetic value. For example, in the financial field, financial institutions often need to design promotional posters to promote financial products, such as wealth management products, insurance products, credit card discounts, etc. These posters need to display key information (such as product name, interest rate, discount content, risk warnings, etc.) in a limited space, while attracting customers' attention and conveying a professional and reliable image. For example, in the medical and health field, hospitals, clinics, public health institutions, etc. need to design promotional posters to promote medical services, health knowledge, public welfare activities, etc. Posters need to clearly display key information (such as medical project introductions, expert consultation information, health lecture times, etc.), while conveying a sense of professionalism and trust to attract the attention of patients and the public.

[0003] Since traditional layout design relies on manual experience or regular templates and is difficult to adapt to diverse design needs, deep learning technology has been introduced into the field of layout generation in recent years. Although poster layout methods based on deep learning can automatically generate layouts, they usually adopt a unified processing method in the face of diverse design needs, lacking the ability to differentiate between different poster design needs, resulting in problems such as waste of computing resources and low layout efficiency. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a poster generation method, device, equipment and medium based on a dynamic neural architecture that can be applied to the medical field, financial technology or other related fields. Its main purpose is to realize dynamic adjustment of the architecture based on the complexity of the design task, ensure the quality of poster generation while reducing computing resource consumption and improving computing efficiency.

[0005] The technical solutions of the present invention are as follows:

[0006] A first aspect of the present invention provides a poster generation method based on a dynamic neural architecture, comprising:

[0007] Acquire original design elements, perform analysis and feature extraction on the original design elements, and obtain corresponding element feature vectors;

[0008] Inputting the element feature vector into a pre-built layout generator, dynamically adjusting the neural network structure of the layout generator according to the feature vector, and obtaining a layout generator with a dynamic architecture;

[0009] Performing encoding processing of the relationship between elements on the element feature vector by the layout generator of the dynamic architecture, and outputting corresponding layout relationship features;

[0010] Performing optimization conversion according to the layout relationship characteristics to generate corresponding optimized layout parameters;

[0011] The original design elements are rendered according to the optimized layout parameters to generate and display a corresponding poster image.

[0012] A second aspect of the present invention provides a poster generation device based on a dynamic neural architecture, comprising:

[0013] A feature extraction module is used to obtain original design elements, analyze and extract features from the original design elements, and obtain corresponding element feature vectors;

[0014] An architecture adjustment module, configured to input the element feature vector into a pre-built layout generator, dynamically adjust the neural network structure of the layout generator according to the feature vector, and obtain a layout generator with a dynamic architecture;

[0015] a layout encoding module, configured to encode the relationship between elements on the element feature vectors through the layout generator of the dynamic architecture, and output corresponding layout relationship features;

[0016] A layout conversion module, configured to perform optimization conversion according to the layout relationship characteristics and generate corresponding optimized layout parameters;

[0017] The rendering and display module is used to render the original design elements according to the optimized layout parameters, and generate and display corresponding poster images.

[0018] A third aspect of the present invention provides a computer device comprising at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned poster generation method based on dynamic neural architecture.

[0021] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned poster generation method based on dynamic neural architecture.

[0022] Beneficial effects: The present invention discloses a poster generation method, apparatus, device and medium based on a dynamic neural architecture. Compared with the prior art, the embodiment of the present invention obtains the original design elements, parses and extracts features of the original design elements, and obtains corresponding element feature vectors; inputs the element feature vectors into a pre-built layout generator, and dynamically adjusts the neural network structure of the layout generator according to the feature vectors to obtain a layout generator with a dynamic architecture; encodes the element feature vectors of the relationship between elements through the layout generator with the dynamic architecture, and outputs corresponding layout relationship features; optimizes and converts the layout relationship features to generate corresponding optimized layout parameters; renders the original design elements according to the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the solutions in the present invention, a brief introduction is given below to the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A schematic diagram of an application environment for the poster generation method based on a dynamic neural architecture provided by an embodiment of the present invention;

[0025] Figure 2 A flowchart of a poster generation method based on a dynamic neural architecture provided by an embodiment of the present invention;

[0026] Figure 3 A flowchart of step S202 in the poster generation method based on dynamic neural architecture provided in an embodiment of the present invention;

[0027] Figure 4 A flowchart of step S203 in the poster generation method based on dynamic neural architecture provided in an embodiment of the present invention;

[0028] Figure 5 A flowchart of step S204 in the poster generation method based on dynamic neural architecture provided by an embodiment of the present invention;

[0029] Figure 6 A schematic diagram of the functional modules of a poster generation device based on a dynamic neural architecture provided by an embodiment of the present invention;

[0030] Figure 7 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and effects of the present invention more clear and distinct, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below with reference to the accompanying drawings.

[0032] The poster generation method based on dynamic neural architecture provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, the system includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0033] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0034] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0035] The server 105 may be a server that provides various services, such as a backend server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The backend server may analyze and process the received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0036] It should be noted that the poster generation method based on the dynamic neural architecture provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the poster generation device based on the dynamic neural architecture provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the poster generation method based on the dynamic neural architecture provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the poster generation device based on the dynamic neural architecture provided in the embodiment of the present invention can generally be set in the server 105.

[0037] It should be understood that the numbers of the above terminal devices, networks and servers are merely illustrative and any number of terminal devices, networks and servers may be provided as required.

[0038] like Figure 2 As shown, the poster generation method based on the dynamic neural architecture provided by the embodiment of the present invention specifically includes the following steps:

[0039] S201: Acquire original design elements, analyze and extract features from the original design elements, and obtain corresponding element feature vectors.

[0040] In this embodiment, when a user designs a poster, the user uploads the original design elements required for the poster design, i.e., the unprocessed basic materials. The original design elements may include text, images, graphics, etc., such as titles, subtitles, body text, main images, background images, icons, lines, shapes, etc. Preferably, for poster designs that require dynamic updates, it is also possible to support receiving real-time original design elements from real-time data streams (such as stock quotes and medical monitoring data) to improve the real-time nature of poster design.

[0041] The input original design elements are parsed by the element parser to extract their basic properties, such as element type (text, image, graphic, etc.), content (text content, image features, etc.), size (width, height), position (initial position), etc., and feature extraction is performed on the parsed results. The parsed elements are converted into feature representations to obtain corresponding element feature vectors. The element feature vectors may include element type, size, content semantics, color, font and other information to form a high-dimensional feature representation. That is, useful information of each original design element is extracted from the element parsing results as input for subsequent layout optimization, thereby reducing redundant information and better reflecting element characteristics.

[0042] For example, in the design of a poster in the financial sector, the original design elements obtained may include bank logos, interest rate information, text descriptions of promotional activities, charts, etc.; in the design of a poster in the healthcare sector, the original design elements obtained may include hospital logos, doctor photos, health tips text, lecture topics, etc. Through parsing and feature extraction, feature vectors such as the position, size, and content semantics of text and image elements are extracted, and the corresponding element feature vectors are obtained to provide data support for layout generation.

[0043] S202: Input the element feature vector into a pre-built layout generator, dynamically adjust the neural network structure of the layout generator according to the feature vector, and obtain a layout generator with a dynamic architecture.

[0044] In this embodiment, a layout generator with a dynamically adjustable neural architecture is pre-built and trained. The layout generator implements element relationship processing and layout parameter prediction through a hybrid architecture (including a gated Transformer encoder, a differentiable neural architecture controller, and a progressive pruning module). After the element feature vector extracted during construction is input into the layout generator, the layout generation prediction is not performed directly. Instead, the internal neural network structure is dynamically adjusted according to the input element feature vector. For example, the number of neurons, the connection method, and the weight of the attention mechanism are dynamically adjusted according to the importance of different elements, thereby obtaining a layout generator with a dynamic architecture that matches the current input element feature vector. Compared with using traditional fixed-architecture neural networks for layout generation, the layout generator with a dynamic architecture can flexibly adapt to design tasks of different complexities, flexibly adjust computing resources, and improve the quality and efficiency of layout generation.

[0045] For example, when designing financial product promotion posters in the financial field, the network structure of the layout generator is dynamically adjusted according to the extracted element feature vectors, and more neurons and attention weights are allocated to more important interest rate information to ensure that it occupies a prominent position in the layout; when designing health promotion lecture posters in the medical and health field, the network structure of the layout generator is dynamically adjusted according to the extracted element feature vectors, such as the higher importance of doctor photos and hospital logos and the higher correlation with some layers in the network, and then the network structure of the layout generator is adjusted to meet the needs of the current design task.

[0046] S203 , performing encoding processing of the relationship between elements on the element feature vector by the layout generator of the dynamic architecture, and outputting corresponding layout relationship features.

[0047] In this embodiment, the relationship between elements is encoded in the element feature vector through the layout generator with an adjusted dynamic architecture. Specifically, the input element feature vector is processed by the gated Transformer encoder in the layout generator to encode the relationship between elements. For example, the mutual influence and importance between elements are calculated through the attention mechanism. After the relationship encoding process, the feature vector describing the layout relationship between elements, namely the layout relationship feature, is output. The layout relationship feature contains information such as the relative position, size ratio, and visual hierarchy between elements, which can be used for subsequent layout parameter generation to achieve adaptive poster layout optimization.

[0048] For example, when designing a promotional poster for a financial product, the layout generator after dynamically adjusting the architecture performs relational encoding on the element feature vectors to determine the relative position relationship between the interest rate information and the bank logo, as well as their layout hierarchy with other elements (such as background images and promotional texts), to ensure the visual balance of the overall layout and the clarity of information communication.

[0049] When designing promotional posters for medical and health lectures, the layout generator with dynamically adjusted architecture performs relational encoding on element feature vectors to determine the relative positional relationship between the doctor's photo and the hospital logo, as well as their layout hierarchy with the health tips text, ensuring that key information is highlighted and the overall layout is aesthetically pleasing.

[0050] S204: Perform optimization conversion according to the layout relationship characteristics to generate corresponding optimized layout parameters.

[0051] In this embodiment, based on the layout relationship features output by the layout generator, alignment is optimized and converted to obtain specific optimized layout parameters. The optimized layout parameters may include the position coordinates, size, rotation angle, etc. of the elements, thereby determining the specific layout of each element in the poster. For example, the layout relationship features can be converted into specific layout parameters and optimized using a differentiable optimization algorithm. During the optimization process, not only the relative positional relationships between elements are considered, but also professional design criteria (such as balance, alignment, aesthetics, etc.) can be further introduced as constraints to ensure the aesthetic quality of the generated layout.

[0052] For example, when designing a promotional poster for a financial product, specific optimized layout parameters are generated and optimized based on the layout relationship characteristics, such as the position coordinates and font size of the interest rate information, to ensure that it is highlighted in the poster while satisfying physical constraints such as not overlapping with other elements.

[0053] When designing promotional posters for medical and health lectures, optimized layout parameters are generated based on layout relationship characteristics. For example, the lecture topic is placed at the top of the poster, the time information is placed at the bottom, the hospital logo is placed in the upper left corner, and the health pictures and doctor photos are placed in the center of the poster. It is ensured that the elements are aligned and do not overlap, etc., thereby generating high-quality optimized layout parameters that comply with design guidelines.

[0054] S205: Render the original design elements according to the optimized layout parameters to generate and display a corresponding poster image.

[0055] In this embodiment, a renderer renders the original design elements based on the generated optimized layout parameters. This means that the optimized layout parameters, such as the layout coordinates, size, and rotation angle, of each original design element are obtained. The rendering engine and real-time rendering methods are then used to draw text, images, graphics, and other elements onto the poster in the specified position, size, and style. The rendered poster image is then visually displayed to the user for easy review and feedback. For example, the final image of a financial product promotion poster can be generated for advertising purposes, or the final image of a medical promotion poster can be generated for hospital promotion or charity events.

[0056] In the above embodiment, the present invention discloses a poster generation method based on a dynamic neural architecture, which obtains original design elements, parses and extracts features from the original design elements, and obtains corresponding element feature vectors; inputs the element feature vectors into a pre-built layout generator, and dynamically adjusts the neural network structure of the layout generator according to the feature vectors to obtain a layout generator with a dynamic architecture; encodes the relationship between elements in the element feature vectors through the layout generator with a dynamic architecture, and outputs corresponding layout relationship features; optimizes and converts the layout relationship features to generate corresponding optimized layout parameters; renders the original design elements according to the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation.

[0057] In one embodiment, step S201 includes:

[0058] Obtaining original design elements, parsing the original design elements, and obtaining corresponding element attribute information;

[0059] Semantic features and visual features are extracted according to the element attribute information, and the extracted semantic features and visual features are uniformly embedded into vectors with the same dimension and then spliced ​​to obtain the element feature vector.

[0060] In this embodiment, when parsing and extracting features from the original design elements, the original design elements are first parsed to extract their basic attributes and structural information, etc., to obtain corresponding element parsing information. For example, for text elements, their content, font, font size and other information are extracted; for image elements, their pixel data, size, color and other information are extracted. Of course, in other embodiments, the text content can also be automatically parsed by, for example, a machine learning model (such as an OCR recognition tool), or an image segmentation algorithm can be used to parse image elements, etc. This embodiment does not limit this. Through automated parsing, the key information of the design elements can be quickly extracted, the design efficiency can be improved, and a high-quality data foundation can be provided for subsequent feature extraction.

[0061] Afterwards, semantic features and visual features are extracted based on the element attribute information. For example, for text elements, semantic features are extracted through pre-trained language models (such as BERT, GPT, etc.), that is, feature vectors used to represent the semantic information of the text content. For image elements, convolutional neural networks (such as ResNet, VGG, etc.) are used to extract visual features, that is, feature vectors used to represent the visual information of the image content, to achieve feature extraction for different types of elements. The extracted semantic features and visual features are aligned, for example, by converting the semantic features and visual features into vectors of the same dimension through linear transformation or other mapping methods, and splicing the semantic feature vectors and visual feature vectors together to form a complete element feature vector. That is, a unified embedding vector is used to ensure that different types of features can be processed in the same feature space, so that multiple types of input elements can be processed efficiently and their subsequent layout modeling is ensured in the same feature space.

[0062] In one embodiment, Figure 3 As shown, step S202 includes:

[0063] S301, inputting the element feature vector into a pre-built layout generator, performing importance evaluation on the original design elements according to the element feature vector, and obtaining an importance score for each original design element;

[0064] S302. Adaptively adjust the number of neurons and the number of attention heads in each layer of the neural network structure of the layout generator according to the importance score of each original design element to obtain a layout generator with a dynamic architecture.

[0065] In this embodiment, when dynamically adjusting the neural network structure of the layout generator, the importance of each original design element is first evaluated based on the input element feature vector. Specifically, a lightweight neural network such as a fully connected layer or a simple Transformer layer can be used to process the original feature vector of each original design element to calculate its importance score s. i :

[0066] s i =σ(W s f i +b s )

[0067] Among them, f i is the eigenvector of the i-th element, W s ∈R 1×d is the learnable weight matrix, b s is the bias term, σ is the sigmoid activation function, and s i The score is normalized to the (0,1) interval, and the importance score si It reflects the relative importance of the element in the overall layout. A larger value indicates that the element should occupy a more prominent position or a larger space in the layout. Assigning an importance score to each design element provides a basis for subsequent dynamic architectural adjustments and ensures that key elements are highlighted in the layout.

[0068] Based on the importance score of each original design element, the number of neurons in each layer of the layout generator is dynamically adjusted. The number of neurons refers to the number of nodes in each layer of the neural network, which affects the computing power and complexity of the model. In addition, the number of attention heads in the gated Transformer encoder is dynamically adjusted, that is, the number of heads of the multi-head attention mechanism in the Transformer architecture, which affects the model's ability to pay attention to different subspaces. By dynamically adjusting the number of neurons and the number of attention heads in the neural network according to the importance score of the element, a layout generator with a dynamic architecture is obtained, which enables the layout generator to adaptively adjust its own structure according to the characteristics of the input elements, better process key elements, save computing resources, and improve the quality and efficiency of layout generation.

[0069] In one embodiment, step S302 includes:

[0070] Calculating a correlation function value between each original design element and each layer based on the element feature vector and the feature embedding of each layer in the neural network structure of the layout generator;

[0071] According to the importance score of each original design element and the correlation function value between each original design element and each layer, the preset number of basic neurons in each layer is scaled to obtain the number of neurons in each layer in the dynamic architecture;

[0072] The average importance score is calculated based on the importance score of each original design element, and the preset number of basic attention heads is adjusted according to the average importance score to obtain the number of attention heads in the dynamic architecture.

[0073] In this embodiment, when specifically adjusting the number of neurons and the number of attention heads, the correlation function value R (e) between each original design element and each layer is calculated based on the element feature vector and the feature embedding of each layer in the neural network structure of the layout generator. i ,l), specifically, we can calculate the cosine angle between two vectors as the correlation function value, that is:

[0074] R(e i ,l)=cos sim (f i ,E l )

[0075] Among them, fi is the element-wise eigenvector of the i-th element, E l is the feature embedding of the lth layer, which is learned through back propagation. According to the importance score of each original design element and the correlation function value between each original design element and each layer, the number of basic neurons preset in each layer is scaled to obtain the number of neurons in each layer in the dynamic architecture. Specifically, the number of neurons in each layer N is dynamically determined by the following formula l :

[0076]

[0077] in is the number of basic neurons in layer l, α is the scaling factor that controls the dynamic adjustment amplitude, R(e i ,l) is the element e i The correlation function value with the lth layer, n is the total number of input elements. The number of neurons in each layer of the layout generator is dynamically determined through scaling processing, making it possible to automatically adjust the network capacity according to the complexity of the design task, avoiding the performance bottleneck of traditional static architectures when processing tasks of different complexities, making the model computing power and resource consumption more compatible with the current design task, and avoiding waste of computing resources.

[0078] When dynamically adjusting the number of attention heads in the gated Transformer encoder in the layout generator, the average importance score is first calculated based on the importance score of each original design element. The preset number of basic attention heads is adjusted based on the average importance score to obtain the number of attention heads in the dynamic architecture. Specifically, the number of attention heads in the gated Transformer encoder is adjusted using the following formula:

[0079]

[0080] Among them, H base is the number of basic attention heads, β is the adjustment coefficient, is the average importance score. The number of attention heads is dynamically adjusted based on the average importance score of all input elements, allowing for flexible adjustments based on the characteristics of the input data. For complex inputs (such as design tasks with multiple important elements), the number of attention heads can be increased to better process the relationships between elements. For simple inputs (such as design tasks with only a few elements), the number of attention heads can be reduced to improve computational efficiency, thereby reducing unnecessary computational resource consumption without sacrificing the quality of processing key elements.

[0081] In one embodiment, Figure 4 As shown, step S203 includes:

[0082] S401, calculating the interaction weights between elements based on the element feature vectors through each attention head in the layout generator of the dynamic architecture;

[0083] S402: Concatenate the interaction weights output by each attention head to obtain a multi-head attention output.

[0084] S403: Input the multi-head attention output into a feedforward neural network for transformation processing, and output corresponding layout relationship features.

[0085] In this embodiment, the layout generator of the dynamic architecture calculates the interaction weights between elements based on the multi-head self-attention mechanism when processing the relationship between elements in the element feature vector. Specifically, each attention head of the gated Transformer encoder in the layout generator calculates the interaction weights between elements based on the input element feature vector. The input element feature vector X is first converted into a query matrix Q, a key matrix K, and a value matrix V through a linear transformation. For each attention head h, the dot product between the query and the key is calculated, and then normalized by the Softmax function to obtain the attention score as the interaction weight:

[0086]

[0087] Among them, H i is the attention score of the i-th attention head, is the learnable parameter matrix, The interaction weights output by each attention head are concatenated to obtain the multi-head attention output:

[0088] Z=Concat(H1,H2,...,H h )W O

[0089] Among them, W O is a learnable parameter matrix used to transform the concatenated multi-head attention outputs into a vector of the same dimension as the input element feature vector. By concatenating and linearly transforming the outputs of multiple attention heads through the multi-head attention mechanism, we can integrate information from different subspaces to generate richer feature representations. We can also learn the interactions between elements from multiple subspaces, providing more comprehensive information about the relationships between elements for subsequent processing.

[0090] After obtaining the multi-head attention output, it is input into a feedforward neural network for linear and nonlinear transformation processing. A feedforward neural network is a neural network containing multiple linear transformation layers and nonlinear activation functions. It is used to further nonlinearly transform the input features to generate more complex feature representations and more accurately describe the layout relationships between elements. The output of the feedforward neural network is the layout relationship feature, which contains information such as the relative position, size ratio, and visual hierarchy between elements. It is used for subsequent layout parameter generation, providing a more accurate feature representation for subsequent layout parameter generation, thereby improving the quality and effectiveness of layout generation.

[0091] In one embodiment, Figure 5 As shown, step S204 includes:

[0092] S501, performing differentiable alignment conversion processing on the layout relationship features to obtain initial layout parameters;

[0093] S502: performing constraint optimization on the initial layout parameters according to preset poster design criteria to generate corresponding optimized layout parameters.

[0094] In this embodiment, the layout relationship features are used as input and converted into initial layout parameters through a differentiable alignment transformation, such as a differentiable Procrustes alignment algorithm. Specifically, the layout relationship features are represented as a relationship matrix Y between elements, and the objective function is used:

[0095]

[0096] where X grid is the target position grid, R is the rotation matrix, t is the translation vector, and the objective function is minimized to solve the rotation matrix and translation vector, thereby converting the aligned results into the initial layout parameters, including the position, size, and rotation angle of the elements.

[0097] In addition, a set of poster design guidelines are preset, including spacing between elements, alignment, hierarchy, and aesthetic standards. For example, it defines that text elements should be located at the top of the poster, image elements should occupy the center, and icons should be evenly distributed around the edges. Based on the design guidelines, the initial layout parameters are further optimized. For example, an optimization objective function is constructed based on each design guideline. The layout parameters are adjusted by minimizing the objective function through an optimization algorithm (such as gradient descent). The initial layout parameters are optimized so that the optimized layout parameters meet the poster design guidelines and have high aesthetic quality, thus achieving adaptive layout optimization.

[0098] In one embodiment, before step S201, the method further includes:

[0099] During the training phase of the layout generator, a layout evaluation is performed on the predicted training poster image to obtain layout feedback information;

[0100] The layout generator is progressively pruned according to the layout feedback information and the number of steps currently trained to obtain a pruned layout generator.

[0101] In this embodiment, during the training phase, the layout generator generates a predicted poster image based on the input training data (including design elements and their feature vectors), and evaluates the generated poster image using predefined evaluation indicators, including aesthetic score, readability score, and element balance. The aesthetic score can evaluate the overall aesthetics of the poster, such as color matching, balance of element layout, etc. The readability score can evaluate the readability of text elements, such as font size, color contrast, and the distinction between text and background. The element balance score can evaluate whether the distribution of elements in the poster is uniform, avoiding overcrowding or sparseness in some areas. After weighting multiple layout evaluation results, the corresponding layout quality indicator Quo is obtained as layout feedback information:

[0102] Quo=λ1A+λ2R+λ3B

[0103] Where A is the aesthetic score, R is the readability score, B is the element balance, and λ1, λ2, and λ3 are learnable weight parameters, the sum of which is 1. Layout evaluation is performed and the evaluation results are quantified as layout feedback information, providing a basis for subsequent model optimization.

[0104] Specifically, when optimizing the model architecture, progressive pruning is performed based on the layout quality indicator Quo and the current number of training steps. That is, the pruning threshold T changes with the training process:

[0105]

[0106] Where t is the current training step number, t max is the total number of steps, γ is the attenuation coefficient, T init =1.

[0107] When the layout quality index Quo>T, no pruning is required. Otherwise, the neural network structure of the layout generator is pruned, that is, unimportant neuron connections are deleted or the number of neurons in certain layers is reduced to ensure that paths that are critical to the processing of key elements are not pruned to optimize the network structure.

[0108] Specifically, the connection probability between neurons can be calculated as the basis for pruning, for example, through softmax calculation:

[0109]

[0110] where θ ij is the connection strength parameter from neuron i to j, τ is the temperature coefficient, which controls the sharpness of the distribution, and N i Is the candidate connection set of neuron i. If the layout quality index Quo is less than or equal to T, then the connection probability p between neurons ij Neurons with probability values ​​less than a preset value, such as 0.5 or other gail values, are pruned, and the corresponding neuron connections are deleted, meaning they will never be connected again. Feedback information is generated through layout evaluation, and progressive pruning is performed based on this feedback information and the current training steps. This allows the layout generator to gradually optimize its own network structure during the training phase, thereby generating high-quality layout solutions with higher efficiency during the inference phase, improving computational efficiency while ensuring that layout quality is not degraded due to pruning.

[0111] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0112] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a poster generation device based on a dynamic neural architecture. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] like Figure 6 As shown, the poster generation device 60 based on the dynamic neural architecture described in this embodiment includes:

[0114] The feature extraction module 601 is used to obtain original design elements, analyze and extract features from the original design elements, and obtain corresponding element feature vectors;

[0115] An architecture adjustment module 602 is configured to input the element feature vector into a pre-built layout generator, dynamically adjust the neural network structure of the layout generator according to the feature vector, and obtain a layout generator with a dynamic architecture;

[0116] A layout encoding module 603 is configured to encode the relationship between elements on the element feature vector using the layout generator of the dynamic architecture, and output corresponding layout relationship features;

[0117] A layout conversion module 604 is configured to perform optimization conversion according to the layout relationship characteristics and generate corresponding optimized layout parameters;

[0118] The rendering and display module 605 is used to render the original design elements according to the optimized layout parameters, and generate and display corresponding poster images.

[0119] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable for describing the poster generation execution process based on dynamic neural architecture than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0120] In one embodiment, the feature extraction module 601 includes:

[0121] An element parsing unit, configured to obtain original design elements, parse the original design elements, and obtain corresponding element attribute information;

[0122] The feature extraction unit is used to extract semantic features and visual features according to the element attribute information, and embed the extracted semantic features and visual features into vectors with the same dimension and then splice them to obtain the element feature vector.

[0123] In one embodiment, the architecture adjustment module 602 includes:

[0124] an importance evaluation unit, configured to input the element feature vector into a pre-built layout generator, perform importance evaluation on the original design elements according to the element feature vector, and obtain an importance score for each original design element;

[0125] The architecture adjustment unit is used to adaptively adjust the number of neurons and the number of attention heads in each layer of the neural network structure of the layout generator according to the importance score of each original design element, so as to obtain a layout generator with a dynamic architecture.

[0126] In one embodiment, the architecture adjustment unit includes:

[0127] a calculation unit, configured to calculate a correlation function value between each original design element and each layer based on the element feature vector and the feature embedding of each layer in the neural network structure of the layout generator;

[0128] A scaling unit is used to scale the number of basic neurons preset in each layer according to the importance score of each original design element and the correlation function value between each original design element and each layer, so as to obtain the number of neurons in each layer in the dynamic architecture;

[0129] An attention head adjustment unit is used to calculate an average importance score based on the importance score of each original design element, and adjust the preset number of basic attention heads based on the average importance score to obtain the number of attention heads in the dynamic architecture.

[0130] In one embodiment, the layout encoding module 603 includes:

[0131] a weight calculation unit, configured to calculate interaction weights between elements based on the element feature vectors through each attention head in the layout generator of the dynamic architecture;

[0132] The splicing unit is used to splice the interaction weights output by each attention head to obtain the multi-head attention output;

[0133] Feature transformation processing is used to input the multi-head attention output into a feedforward neural network for transformation processing and output corresponding layout relationship features.

[0134] In one embodiment, the layout conversion module 604 includes:

[0135] a layout conversion unit, configured to perform a differentiable alignment conversion process on the layout relationship features to obtain initial layout parameters;

[0136] The constraint optimization unit is used to perform constraint optimization on the initial layout parameters according to preset poster design criteria to generate corresponding optimized layout parameters.

[0137] In one embodiment, the apparatus 60 further includes:

[0138] A layout feedback module is used to perform layout evaluation on the predicted training poster image during the training phase of the layout generator to obtain layout feedback information;

[0139] A pruning module is used to perform progressive pruning on the layout generator according to the layout feedback information and the number of steps currently trained to obtain a pruned layout generator.

[0140] In the above embodiment, the present invention discloses a poster generation device based on a dynamic neural architecture, which obtains original design elements, parses and extracts features of the original design elements, and obtains corresponding element feature vectors; inputs the element feature vectors into a pre-built layout generator, and dynamically adjusts the neural network structure of the layout generator according to the feature vectors to obtain a layout generator with a dynamic architecture; encodes the element feature vectors of the relationship between elements through the layout generator with the dynamic architecture, and outputs corresponding layout relationship features; optimizes and converts the layout relationship features to generate corresponding optimized layout parameters; renders the original design elements according to the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation.

[0141] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0142] One or more processors 701 and memory 702, Figure 7 In the description, a processor 701 is used as an example. The processor 701 and the memory 702 can be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0143] The processor 701 is used to complete various control logics of the computer device 70. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination of these components. In addition, the processor 701 can also be any traditional processor, microprocessor or state machine. The processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0144] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as program instructions corresponding to the poster generation method based on a dynamic neural architecture in the embodiments of the present invention. Processor 701 executes the non-volatile software programs, instructions, and modules stored in memory 702 to execute various functional applications and data processing of computer device 70, thereby implementing the poster generation method based on a dynamic neural architecture in the above-mentioned method embodiment.

[0145] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely located relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the poster generation method based on a dynamic neural architecture in any of the above-mentioned method embodiments are executed.

[0146] In the above embodiment, the present invention discloses a computer device, which obtains original design elements, parses and extracts features from the original design elements, and obtains corresponding element feature vectors; inputs the element feature vectors into a pre-built layout generator, and dynamically adjusts the neural network structure of the layout generator according to the feature vectors to obtain a layout generator with a dynamic architecture; encodes the element feature vectors based on the relationship between elements through the layout generator with a dynamic architecture, and outputs corresponding layout relationship features; performs optimization conversion based on the layout relationship features to generate corresponding optimized layout parameters; renders the original design elements based on the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation.

[0147] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the poster generation method based on dynamic neural architecture in any of the above method embodiments are executed.

[0148] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which obtains original design elements, parses and extracts features from the original design elements to obtain corresponding element feature vectors; inputs the element feature vectors into a pre-built layout generator, dynamically adjusts the neural network structure of the layout generator according to the feature vectors, and obtains a layout generator with a dynamic architecture; encodes the element feature vectors for the relationship between elements through the layout generator with a dynamic architecture, and outputs corresponding layout relationship features; optimizes and converts the layout relationship features to generate corresponding optimized layout parameters; renders the original design elements according to the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation.

[0149] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0150] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0151] In summary, the poster generation method, apparatus, device and medium based on dynamic neural architecture disclosed in the present invention include: obtaining original design elements, parsing and extracting features of the original design elements to obtain corresponding element feature vectors; inputting the element feature vectors into a pre-built layout generator, dynamically adjusting the neural network structure of the layout generator according to the feature vectors, and obtaining a layout generator with a dynamic architecture; encoding the element feature vectors with the dynamic architecture layout generator to process the relationship between elements and output corresponding layout relationship features; performing optimization conversion according to the layout relationship features to generate corresponding optimized layout parameters; rendering the original design elements according to the optimized layout parameters to generate and display corresponding poster images. By dynamically adjusting the neural network structure of the layout generator based on the input element features, the network structure can be automatically adjusted and the layout can be optimized according to the complexity of the design task, which can effectively reduce computing resource consumption and improve computing efficiency while ensuring the quality of layout generation.

[0152] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. The storage medium can be a memory, a magnetic disk, a floppy disk, a flash memory, an optical storage device, etc.

[0153] It should be noted that if any software tools or components not developed by our company appear in the examples of this application, they are for illustration purposes only and do not represent actual use. It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the scope of protection of the appended claims.

Claims

1. A poster generation method based on dynamic neural architecture, characterized in that: include: Acquire original design elements, perform analysis and feature extraction on the original design elements, and obtain corresponding element feature vectors; Inputting the element feature vector into a pre-built layout generator, dynamically adjusting the neural network structure of the layout generator according to the feature vector, and obtaining a layout generator with a dynamic architecture; Performing encoding processing of the relationship between elements on the element feature vector by the layout generator of the dynamic architecture, and outputting corresponding layout relationship features; Performing optimization conversion according to the layout relationship characteristics to generate corresponding optimized layout parameters; The original design elements are rendered according to the optimized layout parameters to generate and display a corresponding poster image.

2. The poster generation method based on dynamic neural architecture according to claim 1 is characterized in that The obtaining of original design elements, parsing and feature extraction of the original design elements, and obtaining corresponding element feature vectors include: Obtaining original design elements, parsing the original design elements, and obtaining corresponding element attribute information; Semantic features and visual features are extracted according to the element attribute information, and the extracted semantic features and visual features are uniformly embedded into vectors with the same dimension and then spliced ​​to obtain the element feature vector.

3. The poster generation method based on dynamic neural architecture according to claim 1, characterized in that The step of inputting the element feature vector into a pre-built layout generator and dynamically adjusting the neural network structure of the layout generator according to the feature vector to obtain a layout generator with a dynamic architecture comprises: Inputting the element feature vector into a pre-built layout generator, performing importance evaluation on the original design elements according to the element feature vector to obtain an importance score for each original design element; The number of neurons in each layer and the number of attention heads in the neural network structure of the layout generator are adaptively adjusted according to the importance score of each original design element to obtain a layout generator with a dynamic architecture.

4. The poster generation method based on dynamic neural architecture according to claim 3 is characterized in that Adaptively adjusting the number of neurons and the number of attention heads in each layer of the neural network structure of the layout generator according to the importance score of each original design element to obtain a layout generator with a dynamic architecture includes: Calculating a correlation function value between each original design element and each layer based on the element feature vector and the feature embedding of each layer in the neural network structure of the layout generator; According to the importance score of each original design element and the correlation function value between each original design element and each layer, the preset number of basic neurons in each layer is scaled to obtain the number of neurons in each layer in the dynamic architecture; The average importance score is calculated based on the importance score of each original design element, and the preset number of basic attention heads is adjusted according to the average importance score to obtain the number of attention heads in the dynamic architecture.

5. The poster generation method based on dynamic neural architecture according to claim 1, characterized in that The encoding process of the element feature vectors by the layout generator of the dynamic architecture to determine the relationship between elements and outputting corresponding layout relationship features includes: Calculating interaction weights between elements based on the element feature vectors through each attention head in the layout generator of the dynamic architecture; Concatenate the interaction weights output by each attention head to obtain the multi-head attention output; The multi-head attention output is input into a feedforward neural network for transformation processing, and the corresponding layout relationship features are output.

6. The poster generation method based on dynamic neural architecture according to claim 1, characterized in that The optimizing conversion according to the layout relationship characteristics to generate corresponding optimized layout parameters includes: Performing differentiable alignment transformation processing on the layout relationship features to obtain initial layout parameters; The initial layout parameters are constrained and optimized according to preset poster design criteria to generate corresponding optimized layout parameters.

7. The poster generation method based on dynamic neural architecture according to any one of claims 1 to 6, characterized in that: Before obtaining the original design element, parsing and extracting features from the original design element, and obtaining the corresponding element feature vector, the method further includes: During the training phase of the layout generator, a layout evaluation is performed on the predicted training poster image to obtain layout feedback information; The layout generator is progressively pruned according to the layout feedback information and the number of steps currently trained to obtain a pruned layout generator.

8. A poster generation device based on dynamic neural architecture, characterized in that: include: A feature extraction module is used to obtain original design elements, analyze and extract features from the original design elements, and obtain corresponding element feature vectors; An architecture adjustment module, configured to input the element feature vector into a pre-built layout generator, dynamically adjust the neural network structure of the layout generator according to the feature vector, and obtain a layout generator with a dynamic architecture; a layout encoding module, configured to encode the relationship between elements on the element feature vectors through the layout generator of the dynamic architecture, and output corresponding layout relationship features; A layout conversion module, configured to perform optimization conversion according to the layout relationship characteristics and generate corresponding optimized layout parameters; The rendering and display module is used to render the original design elements according to the optimized layout parameters, and generate and display corresponding poster images.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the poster generation method based on dynamic neural architecture according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the poster generation method based on dynamic neural architecture as described in any one of claims 1-7.

Images