Model training method and device, poster generation method and device, equipment and medium
By evaluating the difficulty of training samples for the poster generation model and dynamically adjusting attention parameters, the problem of redundant computation in traditional training strategies is solved, achieving more efficient training and better generalization capabilities.
Patent Information
- Application Number
- CN202510837152.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-20
- Publication Date
- 2025-10-17
AI Technical Summary
The training strategy of traditional poster generation models treats all samples indiscriminately, resulting in redundant calculations and high training costs, which reduces training efficiency.
By evaluating the difficulty of training samples, the model's attention parameters are dynamically adjusted to optimize the training process, reduce redundant calculations, and improve training efficiency.
The training efficiency of the poster generation model and the processing efficiency in complex design tasks are improved, and the generalization ability of the model is enhanced.
Smart Images
Figure CN120807708A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence and the field of poster design, and in particular to a model training method, a poster generation method, a device, equipment and a medium. BACKGROUND
[0002] As an important form of visual communication, poster design, especially in the business fields of financial technology, medical health and old-age care, has a large audience and great differences among people. Different posters for different people and business fields involve different elements, and the rationality and aesthetics of element layout in different scenarios directly affect the information transmission effect. For example, in the financial technology scenario, the poster design needs to take data visualization, technology symbols and financial icons as core elements, and in the medical health and old-age care scenario, the poster design needs to take medical symbols, humanistic care images and old-age visual elements as core content. Traditional layout design relies on manual experience, and has problems such as low efficiency and strong subjectivity. In recent years, with the development of deep learning, poster automatic layout generation technology based on deep learning has been widely used in poster design to improve the automation level of poster layout generation. However, the traditional training strategy for poster generation model usually treats all samples equally in the training process, while in actual layout tasks, different samples have different contributions to model training. For example, simple samples (such as samples with few elements and regular layout) can quickly converge, while complex samples (such as samples with multiple overlapping elements and asymmetric layout) require more training resources. Therefore, this indiscriminate training method will cause a lot of redundant calculations of the model, not only wasting computing resources, but also significantly increasing the training cost and reducing the overall training efficiency.
[0003] Therefore, how to improve the training efficiency of the poster generation model has become a problem to be solved. SUMMARY
[0004] The embodiments of the present application provide a model training method, a poster generation method, a device, equipment and a medium to solve the problem of how to improve the training efficiency of the poster generation model.
[0005] A training method of a poster generation model, comprising: obtaining a training sample set, each sample in the training sample set comprising an element in a poster image and a layout label of the poster image; training an initial poster generation model according to the training sample set, and obtaining a training prediction result of a corresponding sample output by the initial poster generation model within a preset training period; for any sample, evaluating the difficulty of the sample according to the training prediction result of the sample within the preset training period and the layout label of the sample, and obtaining an evaluation result; According to the evaluation result, the attention parameters of the initial poster generation model on the sample are optimized to obtain an updated poster generation model. The updated poster generation model is used as the initial poster generation model, and the step of training the initial poster generation model according to the training sample set is returned to be executed until the training is completed to obtain a trained poster generation model.
[0006] A poster generation method, comprising: Obtaining the trained poster generation model and the evaluation results of the corresponding samples obtained by the training method of the poster generation model; Obtaining elements in a poster to be generated, performing a similarity comparison between the elements in the poster to be generated and each sample to obtain a comparison result, and determining a layout generation difficulty of the elements in the poster to be generated based on the comparison result and an evaluation result of the corresponding sample; According to the layout generation difficulty, optimizing the attention parameters of the trained poster generation model on the elements in the poster to be generated, to obtain an optimized trained poster generation model; The elements in the poster to be generated are input into the optimized and trained poster generation model to obtain a poster generation result.
[0007] A training device for a poster generation model, comprising: A sample acquisition module, configured to acquire a training sample set, wherein each sample in the training sample set includes an element in a poster image and a layout label of the poster image; A model training module is used to train the initial poster generation model according to the training sample set, and obtain the training prediction results of the corresponding samples output by the initial poster generation model within a preset training cycle; A first evaluation module is configured to evaluate the difficulty of any sample based on the training prediction result of the sample in the preset training cycle and the layout label of the sample to obtain an evaluation result; A loop module is used to optimize the attention parameters of the initial poster generation model on the sample according to the evaluation result to obtain an updated poster generation model, use the updated poster generation model as the initial poster generation model, return to execute the step of training the initial poster generation model according to the training sample set, until the training is completed, and obtain a trained poster generation model.
[0008] A poster generating device, comprising: A model acquisition module is used to obtain the trained poster generation model obtained by the training method of the poster generation model and the evaluation results of the corresponding samples; The comparison module is configured to obtain elements in the to-be-generated poster, compare the elements in the to-be-generated poster with each sample respectively for similarity, obtain comparison results, and determine a layout generation difficulty of the elements in the to-be-generated poster according to the comparison results and evaluation results of the corresponding samples. The first optimization module is configured to optimize attention parameters of the trained poster generation model on the elements in the to-be-generated poster according to the layout generation difficulty, and obtain an optimized trained poster generation model. The poster generation module is configured to input the elements in the to-be-generated poster into the optimized trained poster generation model, and obtain a poster generation result.
[0009] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the training method of the poster generation model or the poster generation method when executing the computer program.
[0010] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the training method of the poster generation model or the poster generation method.
[0011] The training method of the model, the poster generation method, the device, the equipment and the medium are configured to train an initial poster generation model according to a training sample set, obtain training prediction results of corresponding samples output by the initial poster generation model in a preset training period, evaluate a difficulty of each sample according to the training prediction results of the sample in the preset training period and a layout label of the sample, obtain an evaluation result, optimize attention parameters of the initial poster generation model on the sample according to the evaluation result, obtain an updated poster generation model, take the updated poster generation model as the initial poster generation model, return to the step of training the initial poster generation model according to the training sample set, and execute until a trained poster generation model is obtained. According to the evaluation result of the sample, the layout generation difficulty of the elements in the to-be-generated poster is determined, the attention parameters of the trained poster generation model on the elements in the to-be-generated poster are optimized, an optimized trained poster generation model is obtained, the elements in the to-be-generated poster are input into the optimized trained poster generation model, and a poster generation result is obtained.
[0012] Wherein, by evaluating the difficulty of the sample according to the training prediction result and the layout label in different training stages of the model, and dynamically adjusting the attention allocation of the sample by the model according to the evaluation result, the adaptive adjustment of the sample attention according to the training state of the model is realized, the redundant calculation in the model training process is reduced, the reasonable allocation of the model calculation resources is realized, the convergence speed of the model is improved, thereby the training efficiency of the model is improved, and when the poster generation is performed based on the trained poster generation model, the efficiency and the generalization ability of the model in processing the complex design task are also improved. BRIEF DESCRIPTION OF DRAWINGS
[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.
[0014] Figure 1 is an application environment schematic diagram of the training method of the poster generation model in an embodiment of the present application; Figure 2 is a flowchart of the training method of the poster generation model in an embodiment of the present application; Figure 3 is an architecture schematic diagram of the poster generation model in an embodiment of the present application; Figure 4 is another flowchart of the training method of the poster generation model in an embodiment of the present application; Figure 5 is another flowchart of the training method of the poster generation model in an embodiment of the present application; Figure 6 is a flowchart of the poster generation method in an embodiment of the present application; Figure 7 is a schematic diagram of the training device of the poster generation model in an embodiment of the present application; Figure 8 is a schematic diagram of the poster generation device in an embodiment of the present application; Figure 9 is a schematic diagram of the computer device in an embodiment of the present application. DETAILED DESCRIPTION
[0015] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.
[0016] The training method of the poster generation model and the poster generation method provided by the embodiments of the present application can be applied in the application environment as shown in Figure 1 The client and the server communicate through the network, the server provides model training service and poster generation service, and the client triggers the model training task and the poster generation task to the server. The client, also known as the user end, is a program corresponding to the server, which provides local service for the client. The client can be installed on, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.
[0017] For example, in the poster generation task in the field of financial technology, it is necessary to generate a poster image based on specific financial data charts, technological element graphics and financial text. At this time, the poster generation model in the field of financial technology can be trained by the training method of the poster generation model of the present application, and the specific financial data charts, technological element graphics and financial text can be combined with the trained financial technology poster generation model and the evaluation results of the corresponding financial field samples in the training process to generate the financial technology poster image corresponding to the specific financial data charts, technological element graphics and financial text by the poster generation method of the present application.
[0018] In an embodiment, as shown in Figure 2 A training method of a poster generation model is provided, which is applied to Figure 1 The server as shown in The training method comprises the following steps: Step S201: obtaining a training sample set.
[0019] Step S202: training the initial poster generation model according to the training sample set, and obtaining the training prediction result of the corresponding sample output by the initial poster generation model in the preset training period.
[0020] In this embodiment, each sample in the training sample set includes elements in the poster image and a layout label of the poster image. The elements can refer to independent visual units in the poster image, for example, the elements can include financial text, medical popular science text, financial data image, medical image, technology element graph and health element graph in the poster image, and the layout label can refer to the position layout of the elements in the poster image. The initial poster generation model can refer to a pre-set deep learning model to be trained for poster generation. For example, the initial poster generation model can be a model based on a Transformer architecture. The preset training period can refer to a pre-set training period for training the initial poster generation model. The preset training period includes at least one training round. The training prediction result can refer to the training prediction value of the initial poster generation model for the sample layout.
[0021] Specifically, the training sample set is input into the initial poster generation model for training to obtain the training prediction result of the initial poster generation model for the corresponding sample layout in the training round of the preset training period.
[0022] Step S203: For any sample, the difficulty of the sample is evaluated according to the training prediction result of the sample in the preset training period and the layout label of the sample, and an evaluation result is obtained.
[0023] In this embodiment, the evaluation result can refer to a score representing the difficulty of the sample.
[0024] Specifically, for any sample, the training prediction result corresponding to each training round of the sample in the preset training period is determined, and the difficulty of the sample is evaluated according to the training prediction result corresponding to each training round, the layout label of the sample and the number of training rounds in the preset training period, and an evaluation result is obtained.
[0025] Optionally, after obtaining the training prediction result of the corresponding sample, any training prediction result of the sample in the preset training period can be optimized by a preset layout optimization algorithm to obtain an optimized training prediction result. For any sample, the difficulty of the sample is evaluated according to the optimized training prediction result of the sample in the preset training period and the layout label of the sample, and an evaluation result is obtained.
[0026] The preset layout optimization algorithm may refer to a pre-set algorithm for performing layout optimization. For example, the layout optimization algorithm may be an optimization algorithm such as a non-maximum suppression algorithm that solves layout problems such as element overlap. The optimized training prediction result may refer to a layout obtained by optimizing the layout of the training prediction result based on the preset layout optimization algorithm. That is, after obtaining the training prediction result for the corresponding sample, the layout of the training prediction result is optimized based on the preset layout algorithm to solve layout problems such as element overlap, thereby obtaining an optimized training prediction result. The evaluation result is obtained based on the optimized training prediction result and the layout label evaluation.
[0027] Step S204: Based on the evaluation results, the attention parameters of the initial poster generation model on the sample are optimized to obtain an updated poster generation model. The updated poster generation model is used as the initial poster generation model, and the step of training the initial poster generation model according to the training sample set is returned to execute until the training is completed to obtain a trained poster generation model.
[0028] In this embodiment, the attention parameter may refer to the attention weight of the initial poster generation model on the sample, the updated poster generation model may refer to the model after the attention parameter is optimized, and the trained poster generation model may refer to the model that meets the training termination conditions. For example, the training termination conditions may be reaching a preset number of training rounds and the model performance reaching a preset indicator.
[0029] Specifically, for any sample, according to the evaluation result of the sample, the attention parameters of the initial poster generation model on the sample are sparsely processed to obtain the sparse attention parameters, and the initial poster generation model is updated according to the sparse attention parameters corresponding to all samples to obtain an updated poster generation model. The updated poster generation model is used as a new initial poster generation model, and the step of training the initial poster generation model according to the training sample set in the above step S202 is returned to execute, so as to dynamically adjust the model's attention parameters to the sample in different training rounds until the training is completed to obtain a trained poster generation model.
[0030] For example, Figure 3 As shown in FIG, a schematic diagram of the architecture of a poster generation model is provided, including an input module, a layout generation module, and an output module. The overall process of the training method can be as follows: The training sample set is input into the input module. The multimodal elements (such as text, images, etc.) in the input sample are parsed by the element parser, basic attributes are extracted, and the multimodal elements are converted into element vectors by the feature extractor. The self-attention mechanism of the hierarchical Transformer encoder is used to calculate the attention parameters between elements based on element vectors, the attention output is further processed through a feedforward network, the difficulty of the sample is evaluated through the adaptive attention sparsification module to obtain an evaluation result, the element attention parameters are optimized based on the evaluation result through dynamic gating to obtain optimized attention parameters, and the optimized attention parameters are mapped to a latent space through the encoder of the conditional variational autoencoder, and the training prediction result is generated from the latent space by the decoder. The training prediction result is optimized by the layout optimizer of the output module to obtain an optimized training prediction result, and the optimized training prediction result is converted into a visual output by the renderer.
[0031] In the embodiment, the difficulty of the sample is evaluated according to the training prediction result and the layout label in different training stages of the model, and the attention allocation of the sample by the model is dynamically adjusted according to the evaluation result, so that the adaptive adjustment of the sample attention according to the training state of the model is realized, the redundant calculation in the model training process is reduced, the reasonable allocation of the model computing resources is realized, the convergence speed of the model is improved, and thus the training efficiency of the model is improved, and when the poster generation is performed based on the trained poster generation model, the efficiency and generalization ability of the model in processing complex design tasks are also improved.
[0032] In an embodiment, as shown in Figure 4 a training method of a poster generation model is provided, and the difficulty of the sample is evaluated according to the training prediction result of the sample in a preset training period and the layout label of the sample in step S203, and an evaluation result is obtained, including the following steps. Step S401: For any training prediction result of the sample in a preset training period, the intersection over union between the training prediction result and the layout label of the sample is calculated.
[0033] Step S402: According to the intersection over union between all training prediction results of the sample in a preset training period and the layout label, an evaluation result is obtained.
[0034] Specifically, let each sample in the training sample set be k, and the evaluation result be , the training round in the preset training period be N, and the layout label of the sample be , the training prediction result corresponding to the training round be , and the calculation formula of the evaluation result corresponding to the sample can be: , wherein is the intersection over union between the training prediction result and the layout label, , The greater the value is, the greater the difficulty of the sample is.
[0035] In the embodiment, by calculating the intersection over union between the training prediction result and the layout label of the sample in different training stages of the model, the evaluation result of the sample is obtained according to all intersection over unions in a preset training period, the dynamic evaluation of the sample difficulty is realized, the deviation of the static evaluation is avoided, the model can adaptively adjust the attention distribution, thereby reducing the redundant calculation in the model training process, and the training efficiency of the model is improved.
[0036] In an embodiment, as shown in Figure 5 , a training method of a poster generation model is provided, and the step S204 of optimizing the attention parameters of the initial poster generation model on the sample according to the evaluation result to obtain an updated poster generation model includes the following steps: Step S501: For any element in the sample, the attention parameter between the element and any element in the sample except the element is obtained.
[0037] Step S502: According to the evaluation result, the attention parameter between the element and any element in the sample except the element is optimized to obtain an optimized attention parameter. Step S503: According to the optimized attention parameter, the initial poster generation model is updated to obtain an updated poster generation model.
[0038] In the embodiment, the optimized attention parameter can refer to the attention parameter optimized according to the evaluation result.
[0039] Specifically, for any element in the sample, the element is denoted as , any element in the sample except is denoted as , , the attention parameter between and is denoted as , and the optimized attention parameter between and is denoted as , the calculation formula of the optimized attention parameter can be: , wherein is a sigmoid function, is a hyperparameter for controlling the adjustment range of the attention parameter, is a learnable threshold parameter, and the model is used in different training stages through automatic adjustment.
[0040] That is, according to the calculation formula of the optimized attention parameter, the attention parameters between all elements in the sample are optimized to obtain the optimized attention parameters, thereby realizing the optimization of the attention parameters of the initial poster generation model on the sample, and the initial poster generation model is updated according to the optimized attention parameters to obtain an updated poster generation model.
[0041] Optionally, for any element in the sample, an element vector of the element and a vector dimension of the element vector are determined, and an attention parameter between the element and any element other than the element in the sample is obtained according to the element vector of the element, the element vector of any element other than the element in the sample and the vector dimension.
[0042] wherein the element vector can refer to a vector representation of the element. For any element in the sample , the element vector of the element is denoted as , the element vector of any element other than the element in the sample is denoted as , the vector dimension of the element vector and is d, and the calculation formula of the attention parameter between the element and any element other than the element in the sample can be: wherein , , , and are learnable parameter matrices. In the embodiment, by calculating the attention parameter between any element in the sample and any element other than the element, the attention parameter between the elements is optimized according to the evaluation result of the sample, the fine-grained optimization of the attention parameter is realized, the information loss caused by the global coarse-grained optimization of the sample is avoided, the accuracy of the optimization of the attention parameter is improved, and thus the convergence speed of the model is improved and the training efficiency of the model is improved.
[0043] In an embodiment, as shown in , a poster generation method is provided, which is applied to a server as shown in
[0044] , and includes the following steps: Figure 6 Step S601: obtaining a trained poster generation model obtained by a training method of a poster generation model and an evaluation result of a corresponding sample. Figure 1 Step S602: obtaining elements in a to-be-generated poster, performing similarity comparison between the elements in the to-be-generated poster and each sample respectively to obtain comparison results, and determining a layout generation difficulty of the elements in the to-be-generated poster according to the comparison results and the evaluation result of the corresponding sample.
[0045] Step S603: optimizing an attention parameter of the trained poster generation model on the elements in the to-be-generated poster according to the layout generation difficulty, and obtaining an optimized trained poster generation model.
[0046] Step S603: optimizing an attention parameter of the trained poster generation model on the elements in the to-be-generated poster according to the layout generation difficulty, and obtaining an optimized trained poster generation model.
[0047] Step S604: inputting the element in the to-be-generated poster into the trained poster generation model after optimization to obtain a poster generation result.
[0048] In this embodiment, the element in the to-be-generated poster can refer to an element to be laid out to form a poster, the comparison result can refer to a score representing the similarity of the element in the to-be-generated poster to a sample, the layout generation difficulty can refer to a poster generation difficulty of the element in the to-be-generated poster, the trained poster generation model after optimization can refer to a model obtained by optimizing the attention parameters of the trained poster generation model based on the layout generation difficulty, and the poster generation result can refer to a poster generated based on the element in the to-be-generated poster.
[0049] Specifically, for each sample, the element in the to-be-generated poster is compared with the element in the sample poster image in terms of similarity, a similarity score between the element in the to-be-generated poster and the element in the sample poster image is calculated, the similarity score is the comparison result, a sample with the highest similarity score is determined from all samples, all evaluation results corresponding to the sample with the highest similarity are determined, a mean value of all evaluation results corresponding to the sample with the highest similarity is calculated, the mean value is the layout generation difficulty of the element in the to-be-generated poster, the attention parameters of the trained poster generation model on the element in the to-be-generated poster are optimized based on the layout generation difficulty, and a trained poster generation model after optimization is obtained. The element in the to-be-generated poster is input into the trained poster generation model after optimization to obtain a poster generation result.
[0050] For example, if the element in the to-be-generated poster includes a medical image, a health element graph, and medical popular science text, each element is compared with the element in the sample in terms of similarity, the attention parameters of the trained poster generation model on the medical image, the health element graph, and the medical popular science text are adjusted based on the comparison result and the evaluation structure of the corresponding sample, and thus the medical image, the health element graph, and the medical popular science text are input into the trained poster generation model after optimization to generate a medical health poster image formed by the medical image, the health element graph, and the medical popular science text. In this embodiment, the layout generation difficulty is determined based on the similarity comparison result of the element in the to-be-generated poster and the sample, the trained poster generation model is optimized in terms of attention parameters based on the layout generation difficulty, the element in the to-be-generated poster is input into the trained poster generation model after optimization, and a poster generation result is obtained, thereby improving the efficiency of poster generation.
[0051] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined based on its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0052] In an embodiment, a training device of a poster generation model is provided, which corresponds to the training method of the poster generation model in the above-mentioned embodiments. As shown in the figure, the training device of the poster generation model comprises a sample acquisition module 71, a model training module 72, a first evaluation module 73 and a loop module 74. The detailed description of each functional module is as follows: Figure 7 The sample acquisition module 71 is configured to acquire a training sample set, wherein each sample in the training sample set comprises an element in a poster image and a layout label of the poster image. The model training module 72 is configured to train an initial poster generation model according to the training sample set, and acquire a training prediction result of a corresponding sample output by the initial poster generation model within a preset training period. The first evaluation module 73 is configured to evaluate the difficulty of a sample according to a training prediction result of the sample within the preset training period and a layout label of the sample, and obtain an evaluation result. The loop module 74 is configured to optimize an attention parameter of the initial poster generation model on the sample according to the evaluation result, obtain an updated poster generation model, and return to execute the step of training the initial poster generation model according to the training sample set with the updated poster generation model as the initial poster generation model until the training is completed, and obtain a trained poster generation model. Optionally, the first evaluation module 73 comprises:
[0053] A first calculation unit is configured to calculate an intersection over union between a training prediction result of the sample within the preset training period and the layout label of the sample. A second calculation unit is configured to obtain the evaluation result according to the intersection over union between all training prediction results of the sample within the preset training period and the layout label. Optionally, the loop module 74 comprises:
[0054] A parameter acquisition unit is configured to acquire an attention parameter between an element in the sample and any element in the sample except the element. A second optimization unit is configured to optimize the attention parameter between the element and any element in the sample except the element according to the evaluation result, and obtain an optimized attention parameter. A model updating unit is configured to update the initial poster generation model according to the optimized attention parameter, and obtain the updated poster generation model.
[0055] Optionally, the training device for the poster generation model further includes: a vector acquisition module, configured to determine, for any element in the sample, an element vector of the element and a vector dimension of the element vector; The third calculation module is used to obtain the attention parameter between the element and any element except the element based on the element vector of the element, the element vector of any element in the sample except the element, and the vector dimension.
[0056] Optionally, the training device for the poster generation model further includes: A third optimization module is configured to optimize any training prediction result of the sample within the preset training cycle by using a preset layout optimization algorithm to obtain an optimized training prediction result; The second evaluation module is used to evaluate the difficulty of any sample according to the optimization training prediction result of the sample in the preset training cycle and the layout label of the sample to obtain the evaluation result.
[0057] For the specific definition of the training device for the poster generation model, please refer to the definition of the training method for the poster generation model above, which will not be repeated here. The various modules in the training device for the poster generation model can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0058] In one embodiment, a poster generation device is provided, which corresponds to the poster generation method in the above embodiment. Figure 8 As shown, the poster generation device includes a model acquisition module 81, a comparison module 82, a first optimization module 83 and a poster generation module 84. The functional modules are described in detail as follows: A model acquisition module 81 is used to obtain a trained poster generation model obtained by a training method of the poster generation model and evaluation results of corresponding samples; A comparison module 82 is configured to obtain elements in a poster to be generated, perform a similarity comparison between the elements in the poster to be generated and each sample, obtain a comparison result, and determine the layout generation difficulty of the elements in the poster to be generated based on the comparison result and the evaluation result of the corresponding sample; A first optimization module 83 is configured to optimize the attention parameters of the trained poster generation model on the elements in the poster to be generated according to the layout generation difficulty, to obtain an optimized trained poster generation model; The poster generation module 84 is configured to input the elements in the to-be-generated poster into the trained poster generation model to obtain a poster generation result.
[0059] The specific limitations of the poster generation apparatus can refer to the limitations of the poster generation method described above, which will not be repeated here. Each module in the poster generation apparatus described above can be realized by software, hardware, or a combination thereof. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0060] In one embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 9 The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store a training sample set and elements in a to-be-generated poster. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a poster generation model training method.
[0061] In one embodiment, a computer device is provided, which includes a first memory, a first processor, and a computer program stored in the first memory and executable on the first processor. The first processor implements the poster generation model training method in the above embodiments when executing the computer program, such as Figure 2 shown in S201-S204, or Figures 3 to 5 For brevity, the functions of the sample acquisition module 71, the model training module 72, the first evaluation module 83, and the loop module 84 shown in Figure 7 will not be repeated here. The second processor implements the functions of each module / unit in the poster generation model training apparatus in this embodiment when executing the computer program, such as
[0062] In one embodiment, a computer device is provided, which includes a second memory, a second processor, and a computer program stored in the second memory and executable on the second processor. The second processor implements the poster generation method in the above embodiments when executing the computer program, such as Figure 6The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition. Figure 8 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition.
[0063] In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium has stored thereon a computer program. The computer program, when executed by a processor, implements the training method of the poster generation model in the above embodiments, for example Figure 2 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition. Figures 3 to 5 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition. Figure 7 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium has stored thereon a computer program. The computer program, when executed by a processor, implements the training method of the poster generation model in the above embodiments, for example Figure 6 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition. Figure 8 The functions of the model obtaining module 81, the comparison module 82, the first optimization module 83 and the poster generation module 84 shown in FIG. 8 are not repeated here to avoid repetition.
[0064] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0065] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.
[0066] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A training method for a poster generation model, characterized in that: include: Acquire a training sample set, where each sample in the training sample set includes an element in a poster image and a layout label of the poster image; Training the initial poster generation model according to the training sample set, and obtaining the training prediction results of the corresponding samples output by the initial poster generation model within a preset training cycle; For any sample, based on the training prediction result of the sample in the preset training cycle and the layout label of the sample, the difficulty of the sample is evaluated to obtain an evaluation result; According to the evaluation result, the attention parameters of the initial poster generation model on the sample are optimized to obtain an updated poster generation model. The updated poster generation model is used as the initial poster generation model, and the step of training the initial poster generation model according to the training sample set is returned to be executed until the training is completed to obtain a trained poster generation model.
2. The training method of the poster generation model according to claim 1, characterized in that: For any sample, the difficulty of the sample is evaluated according to the training prediction result of the sample in the preset training cycle and the layout label of the sample to obtain an evaluation result, including: For any training prediction result of the sample within the preset training cycle, calculating an intersection-over-union ratio between the training prediction result and the layout label of the sample; The evaluation result is obtained according to the intersection-and-union ratio between all training prediction results of the sample in the preset training cycle and the layout label.
3. The training method of the poster generation model according to claim 1, characterized in that The step of optimizing the attention parameters of the initial poster generation model on the sample according to the evaluation result to obtain an updated poster generation model includes: For any element in the sample, obtaining an attention parameter between the element and any element in the sample other than the element; According to the evaluation result, optimizing the attention parameter between the element and any element in the sample except the element to obtain an optimized attention parameter; The initial poster generation model is updated according to the optimized attention parameters to obtain the updated poster generation model.
4. The training method for the poster generation model according to claim 3, characterized in that: The training method further comprises: For any element in the sample, determining an element vector of the element and a vector dimension of the element vector; An attention parameter between the element and any element except the element is obtained according to the element vector of the element, the element vector of any element in the sample except the element, and the vector dimension.
5. The training method for the poster generation model according to claim 1, characterized in that: After obtaining the training prediction result of the corresponding sample output by the initial poster generation model within the preset training cycle, the method further includes: For any training prediction result of the sample within the preset training cycle, optimizing the training prediction result by a preset layout optimization algorithm to obtain an optimized training prediction result; For any sample, the difficulty of the sample is evaluated according to the optimization training prediction result of the sample in the preset training cycle and the layout label of the sample to obtain the evaluation result.
6. A poster generation method, characterized in that: include:: Obtaining a trained poster generation model and evaluation results of corresponding samples obtained by the training method for the poster generation model according to any one of claims 1 to 5; Obtaining elements in a poster to be generated, performing a similarity comparison between the elements in the poster to be generated and each sample to obtain a comparison result, and determining a layout generation difficulty of the elements in the poster to be generated based on the comparison result and an evaluation result of the corresponding sample; According to the layout generation difficulty, optimizing the attention parameters of the trained poster generation model on the elements in the poster to be generated, to obtain an optimized trained poster generation model; The elements in the poster to be generated are input into the optimized and trained poster generation model to obtain a poster generation result.
7. A training device for a poster generation model, characterized in that: include: A sample acquisition module, configured to acquire a training sample set, wherein each sample in the training sample set includes an element in a poster image and a layout label of the poster image; A model training module is used to train the initial poster generation model according to the training sample set, and obtain the training prediction results of the corresponding samples output by the initial poster generation model within a preset training cycle; A first evaluation module is configured to evaluate the difficulty of any sample based on the training prediction result of the sample in the preset training cycle and the layout label of the sample to obtain an evaluation result; A loop module is used to optimize the attention parameters of the initial poster generation model on the sample according to the evaluation result to obtain an updated poster generation model, use the updated poster generation model as the initial poster generation model, return to execute the step of training the initial poster generation model according to the training sample set, until the training is completed, and obtain a trained poster generation model.
8. A poster generating device, characterized in that: include: A model acquisition module, configured to acquire a trained poster generation model and evaluation results of corresponding samples obtained by the training method for the poster generation model according to any one of claims 1 to 5; a comparison module, configured to obtain elements in a poster to be generated, perform a similarity comparison between the elements in the poster to be generated and each sample, obtain a comparison result, and determine the layout generation difficulty of the elements in the poster to be generated based on the comparison result and the evaluation result of the corresponding sample; A first optimization module is configured to optimize the attention parameters of the trained poster generation model on the elements in the poster to be generated according to the layout generation difficulty, to obtain an optimized trained poster generation model; The poster generation module is used to input the elements in the poster to be generated into the optimized and trained poster generation model to obtain a poster generation result.
9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the training method for the poster generation model according to any one of claims 1 to 5 or the poster generation method according to claim 6 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the training method of the poster generation model according to any one of claims 1 to 5, or the poster generation method according to claim 6.