Poster adaptive layout method and device based on graph attention, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a poster adaptive layout method, device, equipment and medium based on graph attention, and the method comprises the following steps: obtaining all original design elements, and carrying out feature extraction on the original design elements to obtain corresponding element features; taking each original design element as a node, and constructing a directed graph with an initial weight according to the relationship between the original design elements and the element features; calculating a dynamic attention weight between nodes in the directed graph at each time step through a sparse attention mechanism, and updating node embedding to obtain a node updating vector; and performing layout optimization decoding according to the node update vector to generate an optimized layout coordinate of each original design element. By constructing the directed graph of the design elements and updating each node based on the dynamic graph attention, the layout can be optimized according to the dynamic relationship among the elements in the design process, and the self-adaptive optimization of the poster layout is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for poster adaptive layout based on graph attention. Background Art

[0002] As the core carrier of visual communication, poster design needs to maintain a balance between visual aesthetics and functionality while being subject to design constraints. Therefore, higher requirements are placed on the layout optimization of poster design.

[0003] For example, in the financial sector, when financial institutions promote new financial products, credit card services or investment strategies, they need to design attractive promotional posters. When laying out the posters, they need to consider factors such as the characteristics of the financial products and the preferences of the target customer groups, and ensure that the overall layout is in line with the professionalism and seriousness of the financial industry. For example, in the medical and health field, medical research institutions or hospitals need to use posters to display the latest research results, such as clinical trial results, medical paper abstracts, research methods, etc., so when laying out the posters, they need to consider factors such as the importance and relevance of the research content, so as to more effectively convey the research content.

[0004] However, existing poster layout optimization methods have long relied on manual experience or rule-based algorithms. In recent years, deep learning technology has provided new ideas for layout generation. For example, generative adversarial networks (GANs) are used to generate static layout solutions. However, this method still has limitations. It is difficult to achieve layout adaptive processing of multiple poster elements during the design process, which reduces the response timeliness and flexibility of poster layout optimization. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a poster adaptive layout method, device, equipment and medium based on graph attention that can be applied to the medical field, financial technology or other related fields. Its main purpose is to realize the adaptive generation processing of poster layout, which can flexibly adapt to dynamic changes in the design process and improve the response timeliness and flexibility of poster layout optimization.

[0006] The technical solutions of the present invention are as follows:

[0007] A first aspect of the present invention provides a poster adaptive layout method based on graph attention, comprising:

[0008] Acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features;

[0009] Taking each original design element as a node, constructing a directed graph with initial weights according to the relationship between the original design elements and the element features;

[0010] Calculating the dynamic attention weights between nodes in the directed graph at each time step through a sparse attention mechanism and updating the node embedding to obtain a node update vector;

[0011] Layout optimization decoding is performed according to the node update vector to generate optimized layout coordinates of each original design element.

[0012] A second aspect of the present invention provides a poster adaptive layout device based on graph attention, comprising:

[0013] An element acquisition module is used to acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features;

[0014] The graph construction module,semantically treats each original design element as a node and constructs a directed graph with,initial weights based on the relationships between the original design elements and,the features of the elements;

[0015] The node update module uses a sparse attention mechanism to calculate the dynamic attention weights between nodes in the directed graph at each time step and updates the node embedding to obtain a node update vector.

[0016] The layout optimization module semantically performs layout optimization decoding according to the node update vector and generates optimized layout coordinates of each original design element.

[0017] A third aspect of the present invention provides a computer device comprising at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned poster adaptive layout method based on graph attention.

[0020] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned graph attention-based poster adaptive layout method.

[0021] Beneficial effects: The present invention discloses a method, apparatus, device and medium for adaptive poster layout based on graph attention. Compared with the prior art, the embodiments of the present invention obtain all original design elements, extract features from the original design elements, and obtain corresponding element features; take each original design element as a node, and construct a directed graph with initial weights according to the relationship between the original design elements and the element features; calculate the dynamic attention weights between the nodes in the directed graph at each time step through the sparse attention mechanism and update the node embedding to obtain a node update vector; perform layout optimization decoding according to the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and updating nodes based on changes in graph attention, the poster layout can be optimized according to the dynamic relationship between elements in the design process, realizing adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the solutions in the present invention, a brief introduction is given below to the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A schematic diagram of an application environment for the graph attention-based poster adaptive layout method provided by an embodiment of the present invention;

[0024] Figure 2 A flowchart of a poster adaptive layout method based on graph attention provided by an embodiment of the present invention;

[0025] Figure 3 A flowchart of step S202 in the graph attention-based poster adaptive layout method provided in an embodiment of the present invention;

[0026] Figure 4 A flowchart of step S203 in the graph attention-based poster adaptive layout method provided in an embodiment of the present invention;

[0027] Figure 5 A flowchart of step S204 in the graph attention-based poster adaptive layout method provided in an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the functional modules of a poster adaptive layout device based on graph attention provided by an embodiment of the present invention;

[0029] Figure 7A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and effects of the present invention more clear and distinct, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below with reference to the accompanying drawings.

[0031] The poster adaptive layout method based on graph attention provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, the system includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0033] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 may be a server that provides various services, such as a backend server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The backend server may analyze and process the received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0035] It should be noted that the poster adaptive layout method based on graph attention provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the poster adaptive layout apparatus based on graph attention provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the poster adaptive layout method based on graph attention provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the poster adaptive layout apparatus based on graph attention provided in the embodiment of the present invention can generally be set in the server 105.

[0036] It should be understood that the numbers of the above terminal devices, networks and servers are merely illustrative and any number of terminal devices, networks and servers may be provided as required.

[0037] like Figure 2 As shown, the poster adaptive layout method based on graph attention provided by the embodiment of the present invention specifically includes the following steps:

[0038] S201: Acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features.

[0039] In this embodiment, when the user is designing a poster, the user can input the corresponding original design elements into the design system. At this time, the design system receives all the original design elements currently input by the user, including text, images, icons, etc. Feature extraction is performed for each original design element, that is, feature vectors that can be used for subsequent processing are extracted from the original design elements. Specifically, the geometric features (such as position and size), semantic features (such as semantic embedding of text content, visual features of images) and style features (such as fonts and colors) of the original design elements can be extracted. The interactive features of the original design elements (such as whether they are clickable) can also be extracted, so as to obtain the comprehensive element features of each original design element, ensure that the subsequent layout optimization can comprehensively consider the different factors of the elements, and improve the accuracy of adaptive layout optimization.

[0040] For example, when designing promotional posters for financial products in the financial field, the semantic features of the product name (text), the visual features of the product picture (image), and the font and color (style features), etc. are extracted; or, when designing posters for health promotion activities in the medical and health field, the semantic features of the activity theme (text), the visual features of the medical equipment picture (image), and the font and color (style features) of the promotional slogan are extracted to better arrange these elements.

[0041] S202: Take each original design element as a node, and construct a directed graph with initial weights according to the relationship between the original design elements and the element features.

[0042] In this embodiment, the original design elements (text blocks, images, graphics, etc.) are modeled as dynamically evolving graph nodes so that real-time layout optimization can be achieved. That is, each original design element is taken as a node, and the feature vector of the node is obtained by processing the element features of the corresponding original design element. For example, the feature vector of each node is obtained by splicing the element features of different dimensions, etc., so as to comprehensively reflect the characteristics of each node. After the graph nodes are created, the directed edges between each node and the initial weights of each edge are configured according to the relationship between each original design element. The initial weight is used to represent the relationship strength between the nodes at the beginning of the design, thereby obtaining the corresponding directed graph. That is, dynamic associations between elements are established through directed edges and corresponding edge weights, which can explicitly represent the relationship between design elements and transform the traditional static layout into a dynamic graph evolution so that when optimizing the layout, the relationship changes between each element in the design process can be responded to in real time, providing a basis for dynamic adaptive optimization of the layout.

[0043] For example, when designing promotional posters for financial products in the financial field, edges can be established based on factors such as the spatial position and semantic content of the product name and product image, and initial weights can be calculated to construct the initial graph structure; similarly, when designing posters for health promotion activities in the medical and health field, edges can be established based on the spatial position and semantic content of the activity theme and medical equipment images, and the corresponding directed graph can be constructed after calculating the initial weights, so as to adaptively optimize the layout of these elements through dynamic updates of the directed graph.

[0044] S203. Calculate the dynamic attention weights between nodes in the directed graph at each time step through a sparse attention mechanism and update the node embedding to obtain a node update vector.

[0045] In this embodiment, in poster layout design, the relationships between design elements are not static, but change as the design process changes. For example, when a user drags an element, the relationship between the element and other elements will change; or when an element is added or deleted, the semantic relationship of the entire layout will also change; or when the size or position of an element is adjusted, the spatial relationship between elements will change, etc. Based on this dynamic change in element relationships, the dynamic attention weights between nodes are calculated through a sparse attention mechanism at each time step, and the node embedding vectors are updated according to the sparse dynamic attention weights, so that the relationship between design elements can be responded to in real time, allowing the layout to be dynamically optimized and adjusted.

[0046] Specifically, dynamic attention weight refers to the attention weight that changes with time steps. It can capture the relationship changes between elements in real time and adjust the relationship weights between elements according to the changes to achieve more flexible layout adjustment. In addition, this embodiment also uses a sparse attention mechanism to only retain the top-k edges most relevant to the current node for dynamic attention weight calculation. Through sparse processing, it can significantly reduce the computational complexity while maintaining key information and improve layout optimization efficiency.

[0047] For example, when designing a promotional poster for a financial product in the financial field, the user may need to dynamically adjust the position or size of the product image. At this time, the dynamic attention weight can capture the changes in the relationship between the product image and other elements (such as product name, discount information) in real time, and then update the embedding of each node, providing a dynamic data basis for subsequent automatic layout adjustment, ensuring the aesthetics of the poster and the prominence of the information.

[0048] When designing health promotion posters in the medical and health field, users may need to dynamically adjust the position or font size of the activity theme text. Dynamic attention weights can capture the changes in the relationship between the activity theme text and other elements (such as medical equipment pictures, activity time) in real time, and then update the embedding of each node to flexibly adapt to changes in user operations and design needs.

[0049] S204: Perform layout optimization decoding according to the node update vector to generate optimized layout coordinates of each original design element.

[0050] In this embodiment, layout optimization decoding is performed based on the node update vector obtained after dynamic updating, and the final node update vector is decoded into two-dimensional coordinates, thereby generating optimized layout coordinates for each original design element. The layout optimization decoding is a joint optimization combining a differentiable force-guided algorithm and a semantic clustering loss to ensure uniform spacing between elements and highlight key information. In addition, elastic boundary constraints can be used to ensure that elements are always within the effective area of ​​the canvas during the adjustment process to avoid layout overflow. Of course, in other embodiments, other optimization algorithms such as genetic algorithms, simulated annealing algorithms, etc. can also be used, and this embodiment does not limit this.

[0051] In the above embodiment, the present invention discloses a poster adaptive layout method based on graph attention, which obtains all original design elements, extracts features from the original design elements, and obtains corresponding element features; takes each original design element as a node, and constructs a directed graph with initial weights based on the relationship between the original design elements and the element features; calculates the dynamic attention weights between the nodes in the directed graph at each time step through the sparse attention mechanism and updates the node embedding to obtain a node update vector; performs layout optimization decoding based on the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and updating nodes based on changes in graph attention, the poster layout can be optimized based on the dynamic relationship between elements during the design process, realizing adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization.

[0052] In one embodiment, step S201 includes:

[0053] Obtain all original design elements, perform element parsing on the original design elements, and obtain element structured data;

[0054] Feature extraction is performed based on the element structured data to respectively extract the geometric features, semantic features and style features of the original design element.

[0055] In this embodiment, when performing feature extraction, all original design elements that need to be laid out are first obtained, including text blocks, images, icons, graphics, etc. The original design elements are usually stored in the form of files or data structures, such as picture files, text files, HTML elements, SVG elements, etc. These elements can be loaded into the design system through file reading, API calls or user interface interaction. Of course, in addition to local files or APIs, design elements can also be obtained from databases, cloud storage or real-time data streams. This embodiment does not limit this. By supporting multiple data formats and sources, the system can adapt to different design needs.

[0056] The original design elements are parsed into structured data through the element parser. Specifically, the element parser extracts basic information based on the elements. For example, text parsing is performed on text type elements to extract text content, font size, color, alignment, etc.; image parsing is performed on image type elements to extract image path, size, format, etc.; icon parsing is performed on icon type elements to extract icon type, size, color, etc. The parsed information is stored in a structured form, such as JSON, XML or a custom data structure, converting original design elements in different formats into unified structured data for easy subsequent processing.

[0057] Feature extraction is then performed based on the element structured data to extract the geometric, semantic, and style features of the original design elements. That is, element features include geometric, semantic, and style features, describing the characteristics of each original design element from different dimensions. Specifically, geometric features can include position information (extracting the coordinates of the upper left corner of the element), size information (extracting the width and height of the element), and bounding boxes. Semantic features can include text content and image content. For example, for text elements, pre-trained language models (such as BERT and GPT) are used to extract semantic embedding vectors, and for image elements, pre-trained visual models (such as CLIP and ResNet) are used to extract visual feature vectors. Style feature extraction can include font information (extracting the font size and font type of the text), color information (extracting the color of the element, such as RGB values), and alignment (such as left alignment and center alignment). By parsing the original design elements into structured data and extracting their geometric, semantic, and style features, subsequent layout optimization can comprehensively consider the geometric, semantic, and style information of the elements, better understand and process the design elements, and thus achieve more intelligent layout optimization.

[0058] In one embodiment, Figure 3 As shown, step S202 includes:

[0059] S301, creating a node for each original design element, and initializing a feature vector of the corresponding node according to the element characteristics;

[0060] S302 , calculating the spatial proximity and semantic similarity between nodes based on the feature vectors of each node, and initializing the edge weights between nodes based on the spatial proximity and semantic similarity, to construct a directed graph with initial weights.

[0061] In this embodiment, when constructing a directed graph, a node is first created for each original design element. Each node uniquely identifies a design element, for example, by assigning a unique node ID, and the feature vector of the corresponding node is initialized based on the extracted element features. Specifically, when the element features include geometric features, semantic features, and style features, the geometric features, semantic features, and style features can be spliced ​​into a feature vector, and then the spliced ​​feature vector is processed using a multi-layer perceptron (MLP) or other feature fusion methods to obtain the initial feature representation of the node. That is, for an input set containing N elements, when constructing the graph, each element v i The eigenvector of It is composed of three parts:

[0062]

[0063] in, is the normalized geometric feature, is a semantic feature, is the style feature, || represents vector concatenation, and MLP is a multi-layer perceptron.

[0064] Then, a weighted directed graph is constructed based on spatial proximity and semantic relevance. That is, the spatial proximity and semantic similarity between nodes are calculated according to the feature vectors of each node. The edge weights between nodes are initialized according to the spatial proximity and semantic similarity. A directed graph with initial weights is constructed, and then a dynamic graph G is output. t =(V t ,E t ). Specifically, the initial edge set E0 is generated by the dual criteria:

[0065]

[0066] For each pair of nodes i and j, their spatial proximity is measured by calculating the bounding box overlap (IoU, Intersection over Union) of the two elements, b i and b j are the bounding box positions of the two elements respectively; and the semantic relevance of the two elements is measured by calculating the cosine similarity between their semantic features, i.e., sim, τ geo , τ sem is the temperature coefficient, and θ is the empirical threshold. Edge weights are initialized to normalized criterion scores. That is, in the initial stages of constructing a dynamic directed graph, each edge weight is calculated based on the spatial proximity and semantic similarity criteria. These scores are normalized to a value between 0 and 1, and then used as the initial edge weights to construct the initial directed graph. This directed graph explicitly represents the relationships between design elements, providing a foundation for subsequent dynamic updates. Furthermore, the dynamic calculation of spatial proximity and semantic similarity allows for adaptation to changes in design elements and supports dynamic layout adjustments.

[0067] In one embodiment, Figure 4 As shown, step S203 includes:

[0068] S401, at each time step, calculate the query vector, key vector, and value vector of each node based on the current node embedding;

[0069] S402: Prune the designated edge according to the attention weight between the current nodes, and calculate the dynamic attention weight between the nodes of the retained edge according to the query vector, key vector and value vector of each node of the retained edge after pruning;

[0070] S403: Update the embedding representation of each node according to the dynamic attention weights between nodes in the directed graph to obtain a node update vector.

[0071] In this embodiment, dynamic attention propagation of directed graphs is performed based on the captured design changes to dynamically update the relationship weights between nodes and node representations. Specifically, at each time step, the current embedding representation of each node is used as input, and the query vector, key vector, and value vector of each node are calculated based on the current node embedding. For example, for the lth layer, node v i The query vector, key vector, and value vector are:

[0072]

[0073] in are learnable parameters.

[0074] For each pair of nodes (i, j), when calculating the attention score between them, i.e., the dynamic attention weight, the specified edges are pruned according to the attention weight between the current nodes. Only the top-k edges most relevant to each node are retained, and the remaining specified edges, i.e., the edges that do not belong to the top-k edge set, are pruned. For the retained edges, the dynamic attention weight is calculated based on the query vector, key vector, and value vector of the node:

[0075]

[0076] Where d is the dimension of the key vector, The indicator function ensures that only attention is calculated for retained edges, achieving dynamic attention weight calculation for sparse processing. The embedded representation of each node is then updated based on the dynamic attention weights between nodes in the directed graph to obtain a node update vector. Through the dynamic attention mechanism and edge pruning operations, the relationship weights between nodes are dynamically updated, and the node embedding representation is updated. This allows for real-time capture of changes in the relationships between design elements, enabling dynamic layout adjustments while significantly reducing computational effort and improving system efficiency.

[0077] In one embodiment, step S403 includes:

[0078] Calculate the embedding representation of each node at the current time step based on the dynamic attention weights between nodes in the directed graph at the current time step;

[0079] The embedding representation of each node at the current time step is updated with the embedding representation of the historical time step through residual connections across time steps to obtain the corresponding node update vector.

[0080] In this embodiment, when updating node embeddings, for each node i, the value vector is first weighted and summed according to the dynamic attention weight of its retained edge to obtain the embedding representation of the current time step. The weighted summation operation using the dynamic attention weight can effectively integrate information from other nodes into the embedding representation of the current node, thereby reflecting the changes in the relationship between nodes and enabling the layout to be dynamically adjusted.

[0081] Then, the embedding representation of the current time step is updated with the embedding representation of the historical time step through residual connection, and the embedding representation after residual connection is layer normalized, and the layer-normalized embedding representation is used as the final node update vector of the current time step.

[0082]

[0083] Through spatiotemporal jump connections, that is, residual connections across time steps, historical information can be retained and the long-term memory capacity of the model can be enhanced, thereby effectively preventing gradient disappearance and ensuring the stability of the system.

[0084] In one embodiment, Figure 5 As shown, step S204 includes:

[0085] S501, inputting the node update vector and preset boundary information into a trained layout optimizer, wherein the layout optimizer is trained through supervised learning using force-directed loss and semantic clustering loss;

[0086] S502 : The layout optimizer performs joint optimization decoding on the node update vector under the constraint of the preset boundary information to generate optimized layout coordinates of each original design element.

[0087] In this embodiment, when decoding based on the final node update vector, the node update vector is used as input, and the preset boundary information (such as the width and height of the canvas) is input as additional input to the trained layout optimizer. The layout optimizer is a model trained through supervised learning and can generate optimized layout coordinates based on the input node features and boundary information. Specifically, the final coordinates Obtained through joint optimization, the layout optimizer is trained by the force-guided loss and semantic clustering loss during the training phase, where the force-guided loss is the energy function that minimizes the improved FR algorithm:

[0088]

[0089] in For the ideal distance, force-directed loss is used to ensure reasonable spacing between elements to avoid overlap.

[0090] The semantic clustering loss is:

[0091]

[0092] By optimizing the proximity of semantically related elements, semantically similar elements are placed closer together in the layout, thereby enhancing the semantic consistency of the layout. During training, a joint optimization is performed, simultaneously considering the objectives of the force-directed algorithm and the semantic clustering loss, minimizing a comprehensive loss function to adjust the parameters of the layout optimizer and obtain a trained layout optimizer.

[0093] Layout optimization is based on the received node update vectors and preset boundary information. Based on the joint optimization model learned during the training phase, the node update vectors are jointly optimized and decoded within the constraints of the preset boundary information. The position of the original design element represented by each node is adjusted to generate optimized layout coordinates. Specific boundary constraints can be used through differentiable projection operations to bring out-of-bounds elements back into the valid area.

[0094] Through the trained layout optimizer, under the constraints of preset boundary information, the node update vectors are jointly optimized and decoded to generate the optimized layout coordinates of each original design element, thereby generating an aesthetically pleasing and semantically consistent layout. At the same time, it ensures that the layout coordinates of all elements are within the valid range, realizing adaptive generation processing of poster layouts and improving the responsiveness and flexibility of poster layout optimization.

[0095] In one embodiment, after step S204, the method further includes:

[0096] The corresponding original design elements are rendered according to the optimized layout coordinates to generate and display a poster image after the optimized layout.

[0097] In this embodiment, after obtaining optimized layout coordinates based on the dynamically optimized layout solution, the content of the original design elements is visualized. Specifically, each original design element is placed in the corresponding position according to the optimized layout coordinates. The design elements are then rendered onto the canvas using a graphics rendering engine. For example, for text elements, style attributes such as font, size, and color are set, while for image elements, the image size and position are set. All rendered design elements are combined into a complete poster image, which is then displayed to the user. Specifically, this can be displayed through a web interface, desktop application, or mobile application, converting the optimized layout coordinates into a visual poster image. This allows users to intuitively view and adjust the design results, promptly identify and correct layout issues, and improve design efficiency.

[0098] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0099] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a poster adaptive layout device based on graph attention. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0100] like Figure 6 As shown, the poster adaptive layout device 60 based on graph attention described in this embodiment includes:

[0101] The element acquisition module 601 is used to acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features;

[0102] Graph construction module 602, semantically treating each original design element as a node, and constructing a directed graph with initial weights based on the relationship between the original design elements and the element features;

[0103] The node update module 603 calculates the dynamic attention weights between nodes in the directed graph at each time step through a sparse attention mechanism and updates the node embedding to obtain a node update vector;

[0104] The layout optimization module 604 semantically performs layout optimization decoding according to the node update vector to generate optimized layout coordinates of each original design element.

[0105] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable for describing the execution process of poster adaptive layout based on graph attention than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0106] In one embodiment, the element acquisition module 601 includes:

[0107] an acquisition unit, configured to acquire all original design elements, perform element parsing on the original design elements, and obtain element structured data;

[0108] The feature extraction unit is used to perform feature extraction based on the element structured data, and respectively extract the geometric features, semantic features and style features of the original design element.

[0109] In one embodiment, the graph construction module 602 includes:

[0110] A feature initialization unit, configured to create a node for each original design element and initialize a feature vector of the corresponding node according to the element feature;

[0111] The weight initialization unit is used to calculate the spatial proximity and semantic similarity between nodes based on the feature vectors of each node, and initialize the edge weights between nodes based on the spatial proximity and semantic similarity to construct a directed graph with initial weights.

[0112] In one embodiment, the node updating module 603 includes:

[0113] The computation unit is used to compute the query vector, key vector, and value vector of each node based on the current node embedding at each time step;

[0114] Dynamic attention unit, used to prune the specified edge according to the attention weight between the current nodes, and calculate the dynamic attention weight between the nodes of the retained edge according to the query vector, key vector and value vector of each node of the retained edge after pruning;

[0115] A node update unit is used to update the embedding representation of each node according to the dynamic attention weights between nodes in the directed graph to obtain a node update vector.

[0116] In one embodiment, the node updating unit includes:

[0117] An embedding unit, configured to calculate an embedding representation of each node at the current time step based on the dynamic attention weights between nodes in the directed graph at the current time step;

[0118] The residual update unit is used to perform a residual connection update across time steps on the embedded representation of each node at the current time step and the embedded representation of the historical time step to obtain the corresponding node update vector.

[0119] In one embodiment, the layout optimization module 604 includes:

[0120] An input unit, configured to input the node update vector and preset boundary information into a trained layout optimizer, wherein the layout optimizer is trained through supervised learning using force-directed loss and semantic clustering loss;

[0121] A layout optimization unit is configured to perform joint optimization decoding on the node update vectors under the constraint of the preset boundary information by the layout optimizer to generate optimized layout coordinates of each original design element.

[0122] In one embodiment, the apparatus 60 further includes:

[0123] The rendering output module is used to render the corresponding original design elements according to the optimized layout coordinates, and generate and display the poster image after the optimized layout.

[0124] In the above embodiment, the present invention discloses a poster adaptive layout device based on graph attention, which obtains all original design elements, extracts features from the original design elements, and obtains corresponding element features; takes each original design element as a node, and constructs a directed graph with initial weights based on the relationship between the original design elements and the element features; calculates the dynamic attention weights between the nodes in the directed graph at each time step through the sparse attention mechanism and updates the node embedding to obtain a node update vector; performs layout optimization decoding based on the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and updating nodes based on changes in graph attention, the poster layout can be optimized based on the dynamic relationship between elements during the design process, realizing adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization.

[0125] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0126] One or more processors 701 and memory 702, Figure 7 In the description, a processor 701 is used as an example. The processor 701 and the memory 702 can be connected via a bus or other means. Figure 7 The bus connection is taken as an example.

[0127] The processor 701 is used to complete various control logics of the computer device 70. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination of these components. In addition, the processor 701 can also be any traditional processor, microprocessor or state machine. The processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0128] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as program instructions corresponding to the graph attention-based poster adaptive layout method in the embodiments of the present invention. Processor 701 executes the non-volatile software programs, instructions, and units stored in memory 702 to execute various functional applications and data processing of computer device 70, thereby implementing the graph attention-based poster adaptive layout method in the above method embodiment.

[0129] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely located relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned networks include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the poster adaptive layout method based on graph attention in any of the above-mentioned method embodiments are executed.

[0130] In the above embodiment, the present invention discloses a computer device, which obtains all original design elements, extracts features from the original design elements, and obtains corresponding element features; takes each original design element as a node, and constructs a directed graph with initial weights based on the relationship between the original design elements and the element features; calculates the dynamic attention weights between the nodes in the directed graph at each time step through a sparse attention mechanism and updates the node embedding to obtain a node update vector; performs layout optimization decoding based on the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and updating nodes based on changes in graph attention, the poster layout can be optimized based on the dynamic relationship between elements during the design process, realizing adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization.

[0131] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the poster adaptive layout method based on graph attention in any of the above method embodiments are executed.

[0132] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which obtains all original design elements, extracts features from the original design elements, and obtains corresponding element features; takes each original design element as a node, and constructs a directed graph with initial weights based on the relationship between the original design elements and the element features; calculates the dynamic attention weights between the nodes in the directed graph at each time step through a sparse attention mechanism and updates the node embedding to obtain a node update vector; performs layout optimization decoding based on the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and updating nodes based on changes in graph attention, the poster layout can be optimized based on the dynamic relationship between elements during the design process, realizing adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization.

[0133] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0134] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0135] In summary, the method, apparatus, device and medium for adaptive poster layout based on graph attention disclosed in the present invention include: obtaining all original design elements, performing feature extraction on the original design elements, and obtaining corresponding element features; taking each original design element as a node, and constructing a directed graph with initial weights based on the relationship between the original design elements and the element features; calculating the dynamic attention weights between the nodes in the directed graph at each time step through a sparse attention mechanism and updating the node embedding to obtain a node update vector; performing layout optimization decoding based on the node update vector to generate optimized layout coordinates for each original design element. By constructing a directed graph between design elements and performing node updates based on changes in graph attention, the poster layout can be optimized based on the dynamic relationship between elements during the design process, achieving adaptive generation processing of the poster layout, and improving the response timeliness and flexibility of the poster layout optimization.

[0136] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. The storage medium can be a memory, a magnetic disk, a floppy disk, a flash memory, an optical storage device, etc.

[0137] It should be noted that if any software tools or components not developed by our company appear in the examples of this application, they are for illustration purposes only and do not represent actual use. It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the scope of protection of the appended claims.

Claims

1. A poster adaptive layout method based on graph attention, characterized by: include: Acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features; Taking each original design element as a node, constructing a directed graph with initial weights according to the relationship between the original design elements and the element features; Calculating the dynamic attention weights between nodes in the directed graph at each time step through a sparse attention mechanism and updating the node embedding to obtain a node update vector; Layout optimization decoding is performed according to the node update vector to generate optimized layout coordinates of each original design element.

2. The poster adaptive layout method based on graph attention according to claim 1 is characterized in that The step of obtaining all original design elements and performing feature extraction on the original design elements to obtain corresponding element features includes: Obtain all original design elements, perform element parsing on the original design elements, and obtain element structured data; Feature extraction is performed based on the element structured data to respectively extract the geometric features, semantic features and style features of the original design element.

3. The poster adaptive layout method based on graph attention according to claim 1 is characterized in that The method of taking each original design element as a node and constructing a directed graph with initial weights according to the relationship between the original design elements and the element features includes: Creating a node for each original design element and initializing a feature vector of the corresponding node according to the element characteristics; The spatial proximity and semantic similarity between nodes are calculated according to the feature vectors of each node, and the edge weights between nodes are initialized according to the spatial proximity and semantic similarity to construct a directed graph with initial weights.

4. The poster adaptive layout method based on graph attention according to claim 1 is characterized in that The sparse attention mechanism is used to calculate the dynamic attention weights between nodes in the directed graph at each time step and update the node embedding to obtain the node update vector, including: At each time step, the query vector, key vector, and value vector of each node are calculated based on the current node embedding; Prune the specified edges according to the attention weights between the current nodes, and calculate the dynamic attention weights between the nodes of the retained edges based on the query vectors, key vectors, and value vectors of the nodes of each retained edge after pruning; The embedded representation of each node is updated according to the dynamic attention weights between nodes in the directed graph to obtain a node update vector.

5. The poster adaptive layout method based on graph attention according to claim 4 is characterized in that The updating of the embedding representation of each node according to the dynamic attention weights between nodes in the directed graph to obtain a node update vector includes: Calculate the embedding representation of each node at the current time step based on the dynamic attention weights between nodes in the directed graph at the current time step; The embedding representation of each node at the current time step is updated with the embedding representation of the historical time step through residual connections across time steps to obtain the corresponding node update vector.

6. The poster adaptive layout method based on graph attention according to claim 1 is characterized in that The performing layout optimization decoding according to the node update vector to generate optimized layout coordinates of each original design element includes: Inputting the node update vector and preset boundary information into a trained layout optimizer, wherein the layout optimizer is trained through supervised learning using force-directed loss and semantic clustering loss; The layout optimizer performs joint optimization decoding on the node update vector under the constraint of the preset boundary information to generate optimized layout coordinates of each original design element.

7. The poster adaptive layout method based on graph attention according to any one of claims 1 to 6, characterized in that: After performing layout optimization decoding according to the node update vector to generate optimized layout coordinates of each original design element, the method further includes: The corresponding original design elements are rendered according to the optimized layout coordinates to generate and display a poster image after the optimized layout.

8. A poster adaptive layout device based on graph attention, characterized in that: include: An element acquisition module is used to acquire all original design elements, perform feature extraction on the original design elements, and obtain corresponding element features; The graph construction module,semantically treats each original design element as a node and constructs a directed graph with,initial weights based on the relationships between the original design elements and,the features of the elements; The node update module uses a sparse attention mechanism to calculate the dynamic attention weights between nodes in the directed graph at each time step and updates the node embedding to obtain a node update vector. The layout optimization module semantically performs layout optimization decoding according to the node update vector and generates optimized layout coordinates of each original design element.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the poster adaptive layout method based on graph attention as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the poster adaptive layout method based on graph attention as described in any one of claims 1-7.

Images