Poster generation method, poster generation device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a poster generation method, a poster generation device, electronic equipment and a storage medium, belongs to the technical field of artificial intelligence, and is suitable for the field of financial science and technology and the field of digital medical treatment. The method comprises the steps that poster elements and original layout parameters of the poster elements are obtained, feature extraction is conducted on the original layout parameters, a first layout feature with a first feature level and a second layout feature with a second feature level are obtained, and the first feature level is different from the second feature level; carrying out attention calculation according to the first layout feature and the second layout feature to obtain a hierarchical attention feature, carrying out feature coding on the hierarchical attention feature to obtain a layout style parameter, and carrying out parameter updating on the original layout parameter according to the layout style parameter to obtain a target layout parameter, the poster is generated according to the poster elements and the target layout parameters, and the poster generation efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and is applied to the fields of financial technology and digital medicine, and in particular relates to a poster generation method and device, an electronic device and a storage medium. BACKGROUND

[0002] In related technologies, posters can be used to popularize knowledge in related fields and promote business services. For example, in the field of financial technology, financial business posters can be used to popularize financial knowledge and promote financial products. For another example, in the field of digital medicine, medical service posters can be used to popularize health knowledge and disease prevention knowledge and promote medical services. Traditional poster generation methods mainly rely on design experience or template libraries, and cannot quickly generate posters according to business needs, resulting in low efficiency of poster generation. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a poster generation method, a poster generation device, an electronic device and a storage medium, which aims to improve the efficiency of poster generation.

[0004] To achieve the above purpose, a first aspect of the embodiments of the present application provides a poster generation method, which comprises:

[0005] obtaining a poster element and an original layout parameter of the poster element;

[0006] performing feature extraction on the original layout parameter to obtain a first layout feature with a first feature level and a second layout feature with a second feature level, the first feature level and the second feature level being different;

[0007] performing attention calculation according to the first layout feature and the second layout feature to obtain a hierarchical attention feature;

[0008] performing feature encoding on the hierarchical attention feature to obtain a layout style parameter;

[0009] performing parameter updating on the original layout parameter according to the layout style parameter to obtain a target layout parameter;

[0010] generating a poster according to the poster element and the target layout parameter.

[0011] In some embodiments, the feature extraction on the original layout parameter to obtain a first layout feature with a first feature level and a second layout feature with a second feature level comprises:

[0012] performing first feature encoding on the original layout parameter to obtain a first hierarchical feature;

[0013] mapping the first hierarchical feature according to a first preset codebook, to obtain the first layout feature with a first feature level; wherein the first preset codebook has a first codebook level;

[0014] second feature encoding the original layout parameter and the first layout feature, to obtain a second hierarchical feature;

[0015] second feature mapping the second hierarchical feature according to a second preset codebook, to obtain the second layout feature; wherein the second preset codebook has a second codebook level, and the first codebook level is less than the second codebook level.

[0016] In some embodiments, the attention calculation according to the first layout feature and the second layout feature, to obtain a hierarchical attention feature, comprises:

[0017] calculating an attention weight according to the first layout feature and the second layout feature;

[0018] feature mapping the second layout feature according to the attention weight and a preset value weight matrix, to obtain a first layout attention feature;

[0019] feature mapping the first layout feature according to the attention weight and the value weight matrix, to obtain a second layout attention feature;

[0020] determining the hierarchical attention feature according to the first layout attention feature and the second layout attention feature.

[0021] In some embodiments, the attention weight calculation according to the first layout feature and the second layout feature, comprises:

[0022] position encoding the first feature level, to obtain a first position encoding;

[0023] feature fusion of the first layout feature and the first position encoding, to obtain a first encoding feature;

[0024] position encoding the second feature level, to obtain a second position encoding;

[0025] feature fusion of the second layout feature and the second position encoding, to obtain a second encoding feature;

[0026] calculating the attention weight according to the first encoding feature and the second encoding feature.

[0027] In some embodiments, the second layout feature includes a first sub-feature and a second sub-feature, the attention weight includes a first sub-weight and a second sub-weight, and the feature mapping of the second layout feature according to the attention weight and a preset value weight matrix to obtain a first layout attention feature includes:

[0028] feature mapping of the first sub-feature according to the first sub-weight and the value weight matrix to obtain a first reference attention feature;

[0029] feature mapping of the second sub-feature according to the second sub-weight and the value weight matrix to obtain a second reference attention feature;

[0030] feature fusion of the first reference attention feature and the second reference attention feature to obtain the first layout attention feature.

[0031] In some embodiments, the layout style parameter has a style weight, and the parameter updating of the original layout parameter according to the layout style parameter to obtain a target layout parameter includes:

[0032] screening of the layout style parameter according to the style weight to obtain a target style parameter;

[0033] generating a layout parameter distribution according to the target style parameter, and sampling the layout parameter distribution to obtain a sampled layout parameter;

[0034] parameter updating of the original layout parameter according to the sampled layout parameter to obtain the target layout parameter.

[0035] In some embodiments, the parameter updating of the original layout parameter according to the sampled layout parameter to obtain the target layout parameter includes:

[0036] obtaining a parameter number of the sampled layout parameter;

[0037] calculating a parameter selection probability of the sampled layout parameter according to the parameter number, the sampled layout parameter and the layout style parameter;

[0038] screening of the sampled layout parameter according to the parameter selection probability to obtain a reference layout parameter;

[0039] parameter updating of the original layout parameter according to the reference layout parameter to obtain the target layout parameter.

[0040] To achieve the above object, a second aspect of the embodiment of the present application provides a poster generation device, the device comprising:

[0041] An acquisition module is configured to acquire a poster element and original layout parameters of the poster element;

[0042] A feature extraction module is configured to perform feature extraction on the original layout parameters to obtain first layout features with a first feature level and second layout features with a second feature level, the first feature level being different from the second feature level;

[0043] An attention calculation module is configured to perform attention calculation according to the first layout features and the second layout features to obtain hierarchical attention features;

[0044] A feature encoding module is configured to perform feature encoding on the hierarchical attention features to obtain layout style parameters;

[0045] An updating module is configured to perform parameter updating on the original layout parameters according to the layout style parameters to obtain target layout parameters;

[0046] A poster generation module is configured to perform poster generation according to the poster element and the target layout parameters.

[0047] To achieve the above object, a third aspect of embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0048] To achieve the above object, a fourth aspect of embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0049] The poster generation method, the poster generation apparatus, the electronic device and the computer readable storage medium provided in the embodiments of the present application obtain poster elements and original layout parameters of the poster elements, and generate a poster according to the poster elements and the original layout parameters. The original layout parameters are subjected to feature extraction to obtain first layout features with a first feature level and second layout features with a second feature level, and the first feature level and the second feature level are different. By extracting layout features of different levels, the layout business requirements are comprehensively understood, and a poster meeting the business requirements is generated. In order to focus on key features that affect the poster generation and ignore irrelevant features, attention calculation is performed according to the first layout features and the second layout features to obtain hierarchical attention features. The hierarchical attention features are subjected to feature encoding to obtain layout style parameters, so that a layout style more meeting the business requirements is obtained. The original layout parameters are subjected to parameter updating according to the layout style parameters, so that the layout parameters are optimized to obtain target layout parameters, and a high-quality poster is generated. According to the poster elements and the target layout parameters, the poster is generated without relying on design experience or a template library, and the efficiency of the poster generation is improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the poster generation method provided in the embodiments of the present application;

[0051] Figure 2 is a flowchart of step S120 in Figure 1

[0052] Figure 3 is a flowchart of step S130 in Figure 1

[0053] Figure 4 is a flowchart of step S310 in Figure 3

[0054] Figure 5 is a flowchart of step S320 in Figure 3

[0055] Figure 6 is a flowchart of step S150 in Figure 1

[0056] Figure 7 is a flowchart of step S630 in Figure 6

[0057] Figure 8 is a structural schematic diagram of the poster generation apparatus provided in the embodiments of the present application;

[0058] Figure 9 is a hardware structural schematic diagram of the electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION​​​​​​

[0059] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0060] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0062] In the related art, relevant field knowledge can be popularized and business services can be promoted through posters. For example, in the financial technology scenario, financial knowledge can be popularized and financial products can be promoted by making financial business posters. For another example, in the digital medical scenario, health knowledge and disease prevention knowledge can be popularized and medical services can be promoted by making medical service posters. The traditional poster generation method mainly relies on design experience or template library, and cannot quickly generate posters according to business needs, resulting in low efficiency of poster generation.

[0063] Based on this, the embodiments of the present application provide a poster generation method, a poster generation device, an electronic device and a computer readable storage medium, which aim to improve the efficiency of poster generation.

[0064] The poster generation method, the poster generation device, the electronic device and the computer readable storage medium provided by the embodiments of the present application are specifically explained by the following embodiments, and first, the poster generation method in the embodiments of the present application is described.

[0065] The poster generation method provided by the embodiments of the present application relates to the technical field of computers. The poster generation method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like; the server end can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system formed by multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms; and the software can be an application that implements the poster generation method, but is not limited to the above forms.

[0066] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0067] Figure 1 The method in the optional flowchart of the poster generation method provided by the embodiments of the present application can include but is not limited to steps S110 to S160. Figure 1

[0068] Step S110, obtaining a poster element and an original layout parameter of the poster element;

[0069] Step S120, performing feature extraction on the original layout parameter to obtain first layout features with a first feature level and second layout features with a second feature level, the first feature level and the second feature level being different;

[0070] Step S130, performing attention calculation according to the first layout features and the second layout features to obtain hierarchical attention features;

[0071] Step S140, performing feature encoding on the hierarchical attention features to obtain a layout style parameter; ​

[0072] Step S150, parameter updating the original layout parameters according to the layout style parameters to obtain target layout parameters;

[0073] Step S160, poster generation according to the poster elements and the target layout parameters.

[0074] The steps S110 to S160 shown in the embodiments of the present application can comprehensively understand the business requirements by extracting layout features of different levels, obtain layout style parameters more in line with business requirements, and optimize the original layout parameters according to the layout style parameters to obtain target layout parameters, and generate posters according to the poster elements and the target layout parameters. While improving the efficiency of poster generation, the quality of poster generation is improved. It is suitable for application scenarios such as advertisement design, graphic design, UI / UX design, etc. which need to generate high-quality layout efficiently.

[0075] In step S110 of some embodiments, poster elements and original layout parameters of the poster elements are obtained from a poster material library. The poster elements refer to various visual and textual content that constitute a poster, which can include image elements and text elements, etc. The image elements can be illustrations, charts, icons, lines, etc. The text elements can be titles, body content, slogans, etc. Taking the financial technology scene as an example, the image elements can be the interface of a financial application program, a digital payment operation page, etc. The text elements can include intelligent financial management, digital payment, etc. Taking the digital medical scene as an example, the image elements can be the use scene diagram of a smart health detection device, the scene diagram of online consultation, the icon of a medical device, etc. The text elements can include remote medical treatment, health, medical service, etc. Layout parameters are used to define and control the spatial structure and element relevance of poster elements on a poster, and make the layout of poster elements comply with aesthetic rules. The original layout parameters are used to indicate the initial layout of the poster elements. The original layout parameters include the spatial coordinates, size and semantic type of the poster elements. The semantic type description indicates the function and meaning of the poster elements in the overall structure of the poster, such as title text, date text, ticket icon, etc. The original poster can also be obtained from the poster material library, and the original poster is converted into structured data, and the poster elements and the original layout parameters are extracted from the structured data.

[0076] Suppose N poster elements are input L is a collection of N poster elements, e i is the i-th poster element, and the original layout parameters of the poster element e i include spatial coordinates (x i ,y i ), size (w i ,h i ) and semantic type t i . x i and y irespectively, are the length and width of the poster element e i The horizontal coordinate and the vertical coordinate in the poster to be generated are respectively denoted as w i and h i respectively, are the length and width of the poster element e i

[0077] In order to extract fine-grained spatial semantic features, multi-level feature extraction is performed on the original layout parameters by a multi-level encoder to obtain first layout features with a first feature level and second layout features with a second feature level. The multi-level encoder includes at least two levels of encoding layers, the at least two levels including the first feature level and the second feature level, and the encoding layers adopt a network structure of cascading in turn, and the encoding layers can adopt convolutional layers. The encoding layer of each level is used to extract spatial semantic features of a specific granularity. The first feature level and the second feature level are different, and the first feature level is smaller than the second feature level. The first layout features are features extracted by the encoding layer with the first feature level based on the original layout parameters, and the second layout features are features extracted by the encoding layer with the second feature level based on the original layout parameters. Through multi-level feature extraction, layout features of different granularities can be explicitly captured and separated, and feature information from global composition rules to local element relationships can be extracted, so that more structured and coordinated layout designs can be generated.

[0078] Referring to Figure 2 In some embodiments, step S120 can include but is not limited to steps S210 to S240:

[0079] Step S210, first feature encoding is performed on the original layout parameters to obtain first layered features;

[0080] Step S220, first feature mapping is performed on the first layered features according to a first preset codebook to obtain first layout features with a first feature level; wherein the first preset codebook has a first codebook level;

[0081] Step S230, second feature encoding is performed on the original layout parameters and the first layout features to obtain second layered features;

[0082] Step S240, second feature mapping is performed on the second layered features according to a second preset codebook to obtain second layout features; wherein the second preset codebook has a second codebook level, and the first codebook level is smaller than the second codebook level.

[0083] In step S210 of some embodiments, the encoding layer with the first feature level is taken as a first-level encoder, and the original layout parameters of each poster element are first feature encoded by the first-level encoder to extract layout spatial features and obtain first layered features. The first layered features are represented as:

[0084] ​z1 = E1(L),

[0085] wherein z1 is the first hierarchical feature; E1 is the first level encoder; L is the original layout parameter of all poster elements.

[0086] In step S220 of some embodiments, the layout is encoded into a continuous latent vector, and it is difficult to explicitly model the discrete hierarchical structure such as nested bounding boxes, grids, etc., resulting in the generated poster lacking structural controllability. In order to improve the quality of the generated poster, the first hierarchical feature is mapped to a discrete codebook space according to a first preset codebook for first feature mapping, so as to quantize the continuous feature into a codebook vector through the codebook, and obtain a first layout feature with a first feature level. The first preset codebook has a first codebook level, and the first codebook level is equal to the first feature level. The first preset codebook is used as the first level codebook, the first level codebook contains M1 d-dimensional prototype vectors, the feature distance between the first hierarchical feature and each prototype vector is calculated, and the prototype vector with the smallest feature distance is selected as the first layout feature. The calculation process of the first layout feature is represented as:

[0087]

[0088]

[0089] wherein z1 is the first hierarchical feature; is the first layout feature; C1 is the first preset codebook, R represents a real number space; c k is the kth prototype vector; Quantize represents a quantization function for mapping a continuous feature to the nearest codebook vector.

[0090] In step S230 of some embodiments, the encoding layer with the second feature level is used as the second level encoder. The original layout parameters of each poster element and the first layout feature are spliced to obtain a spliced feature. The spliced feature is secondly encoded by the second level encoder, so that the second level encoder can focus on the detailed features that the first level encoder fails to capture, and a second hierarchical feature is obtained.

[0091] The multi-level encoder includes K levels of encoding layers, and K is an integer greater than 1. When k is greater than or equal to 2, each level of encoding layer works in a residual manner, that is, the feature splicing operation is realized in a residual manner, so that the high-level encoder can focus on the detailed features that the previous level fails to capture. A codebook corresponding to the level is set after each level of encoding layer, which is used to quantize the continuous feature. The hierarchical feature of the kth level is represented as:

[0092]

[0093] wherein z kis the hierarchical feature of the kth level; E k is the encoding layer of the kth level; L is the original layout parameter of all poster elements; represents a feature splicing operation; is the layout feature of the k-1th level.

[0094] In step S240 of some embodiments, the second preset codebook has a second codebook level, the first codebook level is smaller than the second codebook level, and the second codebook level is equal to the second feature level. Referring to step S240, the second hierarchical feature is quantized into the nearest codebook vector according to the second preset codebook, to obtain the second layout feature.

[0095] The steps S210 to S240 described above decompose the input layout into feature representations of different scales layer by layer through hierarchical feature extraction and quantization, which can separate the global composition and the local detail features, such as the hierarchical relationship between the title area and the text elements, to represent the hierarchical structure of the layout by using multi-scale discrete latent vectors, and realize the recursive decomposition and reconstruction of the spatial semantic features of the layout elements.

[0096] The first layout feature and the second layout feature are calculated by the decoder to realize cross-level attention, so as to fuse the features of different levels and realize the reconstruction of the poster layout.

[0097] Please refer to Figure 3 In some embodiments, step S130 can include but is not limited to steps S310 to S340:

[0098] Step S310, calculating an attention weight according to the first layout feature and the second layout feature;

[0099] Step S320, performing feature mapping on the second layout feature according to the attention weight and a preset value weight matrix, to obtain a first layout attention feature;

[0100] Step S330, performing feature mapping on the first layout feature according to the attention weight and the value weight matrix, to obtain a second layout attention feature;

[0101] Step S340, determining a hierarchical attention feature according to the first layout attention feature and the second layout attention feature.

[0102] In step S310 of some embodiments, in order to measure the importance of the layout features in the poster generation process, an attention weight between the first layout feature and the second layout feature is calculated. The attention weight is used to measure the degree of mutual influence between different levels of layout features. The greater the attention weight, the greater the degree of feature influence between the first layout feature with the first feature level and the second layout feature with the second feature level. The attention weight includes a first attention weight and a second attention weight, the first attention weight is used to indicate the degree of influence of the second layout feature on the first layout feature, and the second attention weight is used to indicate the degree of influence of the first layout feature on the second layout feature.

[0103] The decoder has a query weight matrix, a key weight matrix, and a value weight matrix. The first query feature is obtained by multiplying the query weight matrix and the first layout feature, the first key feature is obtained by multiplying the key weight matrix and the second layout feature, the similarity between the first query feature and the first key feature is calculated, and the first attention weight is obtained.

[0104] The second query feature is obtained by multiplying the query weight matrix and the second layout feature, the second key feature is obtained by multiplying the key weight matrix and the first layout feature, the similarity between the second query feature and the second key feature is calculated, and the second attention weight is obtained.

[0105] In step S320 of some embodiments, the first attention weight, the value weight matrix, and the second layout feature are multiplied to obtain the first layout attention feature with the first feature level. The first layout attention feature contains features that play a key role in poster generation at the first feature level.

[0106] In step S330 of some embodiments, the second attention weight, the value weight matrix, and the first layout feature are feature mapped to obtain the second layout attention feature with the second feature level. The second layout attention feature contains features that play a key role in poster generation at the second feature level.

[0107] In step S340 of some embodiments, the first layout attention feature and the second layout attention feature are used as the hierarchical attention feature. The hierarchical attention feature contains features that play a key role in poster generation at each level.

[0108] Through the above steps S310 to S340, the key features of each level can be obtained to optimize the original layout parameters based on these key features.

[0109] Please refer to Figure 4 In some embodiments, step S310 can include but is not limited to steps S410 to S450:

[0110] Step S410, position encoding is performed on the first feature hierarchy to obtain first position encoding;

[0111] Step S420, feature fusion is performed on the first layout feature and the first position encoding to obtain first encoded feature;

[0112] Step S430, position encoding is performed on the second feature hierarchy to obtain second position encoding;

[0113] Step S440, feature fusion is performed on the second layout feature and the second position encoding to obtain second encoded feature;

[0114] Step S450, attention weight is calculated according to the first encoded feature and the second encoded feature.

[0115] In step S410 of some embodiments, a hierarchical position encoding function is introduced to embed the position information of the poster element in the poster into the decoder, so that the position information is better understood and utilized to generate a reasonable layout. The first feature hierarchy is position encoded by the hierarchical position encoding function. Specifically, for each feature dimension, if the dimension index value of the feature dimension is even, the first feature hierarchy and the feature dimension are calculated by a sine function to obtain the position encoding of the feature dimension. If the dimension index value of the feature dimension is odd, the first feature hierarchy and the feature dimension are calculated by a cosine function to obtain the position encoding of the feature dimension. The position encoding of each feature dimension is taken as the first position encoding. The number of feature dimensions of the first position encoding is equal to the first layout feature. The hierarchical position encoding function is represented as:

[0116]

[0117] wherein k represents the kth hierarchy, which can be the first feature hierarchy; i is the dimension index value of the ith feature dimension, i is an integer greater than or equal to 0; d is the number of feature dimensions of the first layout feature; sin represents a sine function; and cos represents a cosine function.

[0118] In step S420 of some embodiments, the first layout feature and the first position encoding are added in feature to obtain the first encoded feature with the first feature hierarchy.

[0119] In step S430 of some embodiments, referring to step S410, the second feature hierarchy is position encoded by the hierarchical position encoding function to obtain the second position encoding. The number of feature dimensions of the second position encoding is equal to the second layout feature.

[0120] In step S440 of some embodiments, the second layout feature and the second position encoding are added in feature to obtain the second encoded feature with the second feature hierarchy.

[0121] In step S450 of some embodiments, the query weight matrix is multiplied by the first encoded feature to obtain a first query feature, the key weight matrix is multiplied by the second encoded feature to obtain a first key feature, the similarity between the first query feature and the first key feature is calculated to obtain a first attention weight.

[0122] The query weight matrix is multiplied by the second encoded feature to obtain a second query feature, the key weight matrix is multiplied by the first encoded feature to obtain a second key feature, and the similarity between the second query feature and the second key feature is calculated to obtain a second attention weight.

[0123] Referring to step S320, the first attention weight obtained in step S450, the value weight matrix, and the second encoded feature are multiplied to obtain a first layout attention feature. The second attention weight obtained in step S450, the value weight matrix, and the first encoded feature are multiplied to obtain a second layout attention feature.

[0124] In some embodiments, the decoder receives K levels of layout features For each level, the level k is positionally encoded by a level position encoding function to obtain a position encoding of the level k. The layout feature of the level k and the position encoding of the level k are added to obtain an encoded feature of the level k. The encoded feature of the level k is denoted as:

[0125]

[0126] wherein, is the layout feature of the level k; is the encoded feature of the level k; and LevelPE is the level position encoding function.

[0127] The influence degree of the layout feature of the level j on the layout feature of the level i, i.e., the attention weight, is denoted as:

[0128]

[0129] wherein, α ij is the attention weight of the layout feature of the level j on the layout feature of the level i; T represents a transposition operation; represents the encoded feature of the level i; represents the encoded feature of the level j; d represents the feature dimension number of the layout feature; softmax is used to calculate the similarity between the query feature and the key feature; W Q represents the query weight matrix; W K represents the key weight matrix.

[0130] Through the above steps S410 to S450, the attention weights between the cross-level layout features can be obtained, so as to fuse the cross-level layout features based on the attention weights.

[0131] Referring to Figure 5 In some embodiments, step S320 can include, but is not limited to, steps S510-S530:

[0132] In step S510, the first sub-feature is mapped according to the first sub-weight and the value weight matrix to obtain a first reference attention feature.

[0133] In step S520, the second sub-feature is mapped according to the second sub-weight and the value weight matrix to obtain a second reference attention feature.

[0134] In step S530, the first reference attention feature and the second reference attention feature are fused to obtain a first layout attention feature.

[0135] In step S510 of some embodiments, the second layout feature includes the first sub-feature and the second sub-feature, the second feature level includes the first sub-level and the second sub-level, the first sub-level is the level of the first sub-feature, the second sub-level is the level of the second sub-feature, and the first sub-level is smaller than the second sub-level. The attention weight includes a first attention weight, the first attention weight is used to indicate the influence degree of the second layout feature on the first layout feature based on the first layout feature to calculate the query feature, and the first attention weight includes a first sub-weight and a second sub-weight. The first sub-weight and the second sub-weight are respectively used to indicate the influence degree of the first sub-feature and the second sub-feature on the first layout feature. The first sub-weight and the second sub-weight can be calculated based on the above attention calculation formula. The first sub-weight, the value weight matrix and the first sub-feature are multiplied to obtain the first reference attention feature.

[0136] In step S520 of some embodiments, the second sub-weight, the value weight matrix and the second sub-feature are multiplied to obtain the second reference attention feature.

[0137] In step S530 of some embodiments, the first reference attention feature and the second reference attention feature are added to obtain the first layout attention feature.

[0138] The second layout feature can also be replaced by the second encoding feature to perform steps S510-S530 to obtain the first layout attention feature. The layout attention feature of level i is represented as:

[0139]

[0140] wherein, is the layout attention feature of level i; α ij is the attention weight of the feature of level j on the feature of level i; W V is the value weight matrix; For the encoding feature of the level j, j is any level not equal to i.

[0141] Through the steps S510 to S530, the features of different levels can be allowed to affect each other, and the global consistency of the generated layout is ensured.

[0142] In step S140 of some embodiments, the hierarchical attention features are encoded by a mixture density network (MDN) to convert discrete latent variables into continuous layout parameter outputs, and layout style parameters are obtained. The mixture density network adopts a mixture Gaussian model including M Gaussian models to generate diversified layout schemes. Based on the mixture density network, layout style parameters of the M Gaussian models can be determined. The layout style parameters are used to indicate the distribution of the layout parameters, and the layout style parameters include a mean vector, a variance vector, and a style weight. The style weight is a model weight of the Gaussian model in the mixture Gaussian model.

[0143] Referring to Figure 6 In some embodiments, step S150 can include but is not limited to steps S610 to S630:

[0144] In step S610, the layout style parameters are filtered according to the style weight, and target style parameters are obtained.

[0145] In step S620, a layout parameter distribution is generated according to the target style parameters, and a sampling layout parameter is obtained by sampling the layout parameter distribution.

[0146] In step S630, the original layout parameters are updated according to the sampling layout parameters, and target layout parameters are obtained.

[0147] In step S610 of some embodiments, the style weight can reflect the probability of generating a poster using a certain style. The greater the style weight, the greater the probability of selecting the layout style parameter. In order to generate a poster that meets the business requirements, the layout style parameter with the largest style weight is selected as the target style parameter.

[0148] In step S620 of some embodiments, a Gaussian distribution is generated according to the mean vector and the variance vector in the target style parameters, and a layout parameter distribution is obtained. In order to generate diversified poster layouts, the layout parameter distribution is randomly sampled to obtain a sampling layout parameter. The sampling layout parameter includes the spatial coordinates, the size, and the semantic type of the poster elements.

[0149] Assuming that the mean vector is {x1, x2, x3}, the variance vector is {y1, y2, y3}, x1, x2, and x3 are the means for the spatial coordinates, the size, and the semantic type, respectively, and y1, y2, and y3 are the variances for the spatial coordinates, the size, and the semantic type, respectively. The Gaussian distribution for the spatial coordinates, the size, and the semantic type can be determined according to (x1, y1), (x2, y2), and (x3, y3), respectively. Random sampling is performed on the Gaussian distribution for the spatial coordinates, the size, and the semantic type to obtain the sampling layout parameters of each poster element.

[0150] In step S630 of some embodiments, in order to improve the quality of poster generation, the original layout parameters are updated to the sampling layout parameters to perform layout optimization on the original layout parameters to obtain target layout parameters.

[0151] Through steps S610 to S630 described above, while improving the quality of poster generation, posters that meet business requirements and are diversified can be quickly generated.

[0152] Please refer to Figure 7 In some embodiments, step S630 can include but is not limited to steps S710 to S740:

[0153] In step S710, the number of parameters of the sampling layout parameters is obtained.

[0154] In step S720, according to the number of parameters, the sampling layout parameters, and the layout style parameters, the parameter selection probability of the sampling layout parameters is calculated.

[0155] In step S730, the sampling layout parameters are filtered according to the parameter selection probability to obtain reference layout parameters.

[0156] In step S740, the original layout parameters are updated according to the reference layout parameters to obtain target layout parameters.

[0157] In step S710 of some embodiments, there can be multiple sampling layout parameters for the same poster element, and in order to improve the quality of poster generation, the sampling layout parameters need to be filtered. Specifically, the number of parameters of the sampling layout parameters of the poster element is obtained to obtain the number of parameters.

[0158] In step S720 of some embodiments, if the number of parameters is 1, the original layout parameters are updated according to the sampling layout parameters to obtain target layout parameters. If the number of parameters is greater than 1, the probability that the sampling layout parameters are selected in the poster generation process is calculated according to the sampling layout parameters and the layout style parameters to obtain the parameter selection probability. The parameter selection probability is expressed as:

[0159]

[0160] wherein, is a hierarchical attention feature, containing layout attention features of each hierarchy; e i represents the i-th poster element; M is the number of Gaussian models; m represents the m-th Gaussian model; π m is the style weight of the m-th Gaussian model; μ m and∑ m is the layout style parameter of the m-th Gaussian model, μ m represents the mean vector,∑ m represents the variance vector; is a Gaussian distribution generated based on the mean vector and the variance vector.

[0161] The sampled layout parameter is substituted into each Gaussian distribution constructed by the layout style parameter to obtain the probability output by each Gaussian distribution. The parameter selection probability is obtained by weighting calculation according to the style weight of each Gaussian distribution and the probability output by the corresponding Gaussian distribution. The parameter selection probability is used to indicate the probability of the sampled layout parameter being selected in the poster generation process.

[0162] In step S730 of some embodiments, the sampled layout parameter with the maximum parameter selection probability is selected as the reference layout parameter.

[0163] In step S740 of some embodiments, the original layout parameter is parameter updated according to the reference layout parameter to update the original layout parameter to the reference layout parameter, obtaining the target layout parameter.

[0164] Through the above steps S710 to S740, a more optimal target layout parameter can be obtained, thereby improving the efficiency and quality of poster generation.

[0165] In step S160 of some embodiments, the poster is generated according to the poster elements and the target layout parameters of the poster elements. The poster elements can be arranged and designed on the canvas according to the target layout parameters, and the poster is generated.

[0166] Consider the scenario of poster design for a financial product launch, the original layout parameters of each poster element are (product main photo, position: (0.1, 0.1), size: (0.3, 0.3)), (logo, position: (0.6, 0.6), size: (0.2, 0.2)), (title text, position: (0.4, 0.4), size: (0.2, 0.1)), (date text, position: (0.7, 0.8), size: (0.1, 0.05)), (ticket icon, position: (0.9, 0.9), size: (0.1, 0.1)). After the method of the present application is processed, the optimized layout is: all elements are aligned along the vertical axis, the main visual elements (photo and logo) form a balanced visual focus, the text information is arranged according to importance level, and the ticket icon is highlighted. The method of the present application can convert the initial layout of chaos into a professional-level visual design while maintaining the logical association and visual balance of all elements.

[0167] See Figure 8 The present application also provides a poster generation device, which can implement the poster generation method described above, and the poster generation device comprises:

[0168] The acquisition module 810 is configured to acquire poster elements and original layout parameters of the poster elements.

[0169] The feature extraction module 820 is configured to perform feature extraction on the original layout parameters to obtain first layout features with a first feature level and second layout features with a second feature level, the first feature level being different from the second feature level.

[0170] The attention calculation module 830 is configured to perform attention calculation according to the first layout features and the second layout features to obtain hierarchical attention features.

[0171] The feature encoding module 840 is configured to perform feature encoding on the hierarchical attention features to obtain layout style parameters.

[0172] The update module 850 is configured to perform parameter updating on the original layout parameters according to the layout style parameters to obtain target layout parameters.

[0173] The poster generation module 860 is configured to perform poster generation according to the poster elements and the target layout parameters.

[0174] The specific implementation of the poster generation device is basically the same as the specific embodiments of the poster generation method described above, and will not be repeated here.

[0175] The embodiment of the present application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the poster generation method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0176] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0177] The processor 910 can be implemented in the form of a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0178] The memory 920 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 920 and are called and executed by the processor 910 to implement the poster generation method of the embodiments of the present application.

[0179] The input / output interface 930 is used to realize information input and output.

[0180] The communication interface 940 is used to realize the communication interaction between the device and other devices, and can realize communication through a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0181] The bus 950 transmits information between various components (such as the processor 910, the memory 920, the input / output interface 930, and the communication interface 940) of the device.

[0182] The processor 910, the memory 920, the input / output interface 930, and the communication interface 940 are connected to each other through the bus 950 to realize communication connection within the device.

[0183] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the poster generation method.

[0184] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0185] The poster generation method and device, electronic equipment and computer storage medium provided by the embodiments of the present application can comprehensively understand the business requirements by extracting layout features of different levels, obtain layout style parameters more in line with the business requirements, and optimize the original layout parameters according to the layout style parameters to obtain target layout parameters, and generate a poster according to the poster elements and the target layout parameters, thereby improving the efficiency of poster generation and the quality of poster generation.

[0186] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0187] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures shown, or combine certain steps or different steps.

[0188] The device embodiments described above are only schematic, and the units illustrated as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0189] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0190] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a given step or its integral presence in the process, method, system, article, or apparatus having been made with a wider scope. The use of notation such as "first", "second", "third", etc. does not generally limit the areas, but is used to connect like elements or to distinguish one claim from another. These terms can be used interchangeably when appropriate. Terms concerning the relative position of elements can be interpreted such that their use adheres to their normal meaning, but they can also be interpreted to mean the opposite according to specific claims.

[0191] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0192] In several embodiments provided by the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0193] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the application.

[0194] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0195] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0196] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A poster generation method, characterized in that: The method comprises: Obtaining a poster element and original layout parameters of the poster element; Performing feature extraction on the original layout parameters to obtain a first layout feature having a first feature level and a second layout feature having a second feature level, wherein the first feature level and the second feature level are different; Performing attention calculation based on the first layout feature and the second layout feature to obtain a hierarchical attention feature; Performing feature encoding on the hierarchical attention features to obtain layout style parameters; updating the original layout parameters according to the layout style parameters to obtain target layout parameters; A poster is generated according to the poster elements and the target layout parameters.

2. The method according to claim 1, characterized in that The extracting features of the original layout parameters to obtain first layout features having a first feature level and second layout features having a second feature level includes: Performing a first feature encoding on the original layout parameters to obtain a first hierarchical feature; Performing a first feature mapping on the first hierarchical feature according to a first preset codebook to obtain the first layout feature having a first feature level; wherein the first preset codebook has a first codebook level; Performing a second feature encoding on the original layout parameters and the first layout features to obtain a second hierarchical feature; Performing a second feature mapping on the second hierarchical feature according to a second preset codebook to obtain the second layout feature; wherein the second preset codebook has a second codebook level, and the first codebook level is smaller than the second codebook level.

3. The method according to claim 1, characterized in that The performing attention calculation according to the first layout feature and the second layout feature to obtain a hierarchical attention feature includes: Calculating an attention weight according to the first layout feature and the second layout feature; Performing feature mapping on the second layout feature according to the attention weight and a preset value weight matrix to obtain a first layout attention feature; Performing feature mapping on the first layout feature according to the attention weight and the value weight matrix to obtain a second layout attention feature; The hierarchical attention feature is determined based on the first layout attention feature and the second layout attention feature.

4. The method according to claim 3, characterized in that The calculating the attention weight according to the first layout feature and the second layout feature includes: Performing position encoding on the first feature level to obtain a first position code; Performing feature fusion on the first layout feature and the first position code to obtain a first coding feature; performing position encoding on the second feature level to obtain a second position code; Performing feature fusion on the second layout feature and the second position code to obtain a second code feature; Calculating the attention weight according to the first encoding feature and the second encoding feature.

5. The method according to claim 3, characterized in that The second layout feature includes a first sub-feature and a second sub-feature, the attention weight includes a first sub-weight and a second sub-weight, and feature mapping is performed on the second layout feature according to the attention weight and a preset value weight matrix to obtain a first layout attention feature, including: Performing feature mapping on the first sub-feature according to the first sub-weight and the value weight matrix to obtain a first reference attention feature; Performing feature mapping on the second sub-feature according to the second sub-weight and the value weight matrix to obtain a second reference attention feature; The first reference attention feature and the second reference attention feature are subjected to feature fusion to obtain the first layout attention feature.

6. The method according to any one of claims 1 to 5, characterized in that The layout style parameters have style weights, and the updating of the original layout parameters according to the layout style parameters to obtain target layout parameters includes: Filtering the layout style parameters according to the style weights to obtain target style parameters; generating a layout parameter distribution according to the target style parameters, and sampling the layout parameter distribution to obtain sampled layout parameters; The original layout parameters are updated according to the sampled layout parameters to obtain the target layout parameters.

7. The method according to claim 6, characterized in that The updating of the original layout parameters according to the sampled layout parameters to obtain the target layout parameters includes: Obtaining the number of parameters of the sampling layout parameter; Calculating a parameter selection probability of the sampled layout parameters according to the parameter quantity, the sampled layout parameters, and the layout style parameters; Screening the sampled layout parameters according to the parameter selection probability to obtain reference layout parameters; The original layout parameters are updated according to the reference layout parameters to obtain the target layout parameters.

8. A poster generating device, characterized in that: The device comprises: An acquisition module, used to acquire the poster element and the original layout parameters of the poster element; a feature extraction module, configured to perform feature extraction on the original layout parameters to obtain a first layout feature having a first feature level and a second layout feature having a second feature level, wherein the first feature level and the second feature level are different; an attention calculation module, configured to perform attention calculation based on the first layout feature and the second layout feature to obtain a hierarchical attention feature; A feature encoding module, configured to perform feature encoding on the hierarchical attention features to obtain layout style parameters; An updating module, configured to update the original layout parameters according to the layout style parameters to obtain target layout parameters; A poster generation module is used to generate a poster according to the poster elements and the target layout parameters.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images