Method, device, computer equipment and storage medium for generating Chinese art posters

Abstract

The present invention discloses a method, device, computer equipment and storage medium for generating Chinese art posters. The method includes: based on a background image and Chinese propaganda text, generating Chinese layout information of Chinese propaganda text in the background image; based on the Chinese layout information, performing image positioning and segmentation on the background image to obtain a local image of the area where the Chinese propaganda text is located, and based on the Chinese propaganda text and the local image, using a target prediction model to perform text style prediction to obtain text style information; based on the text style information, performing style and graphic conversion processing on the Chinese propaganda text to generate a target Chinese graphic; based on the Chinese layout information, integrating the target Chinese graphic into the background image to obtain a Chinese art poster. In this embodiment, the font accuracy and style diversity in the Chinese art poster can be improved, the visual effect of the Chinese art poster can be effectively improved, and the generation efficiency of the Chinese art poster is improved while taking into account its generation quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, computer equipment and storage medium for generating a Chinese art poster. Background Art

[0002] As one of the important means of information dissemination, artistic posters are widely used in advertising and entertainment industries. In order to improve the efficiency of information dissemination, the demand for automatically creating posters based on the prompt text provided by users is becoming more and more urgent. For this reason, the text-to-image (Text2Image) model was born to generate posters.

[0003] Due to the complexity of Chinese characters and font styles, when traditional text-to-image models generate posters with Chinese characters, they cannot correctly generate Chinese characters with artistic fonts, and the generated Chinese art posters are not good. However, the correctness and style diversity of Chinese characters are crucial for Chinese art posters. In order to generate high-quality Chinese art posters with good visual effects and accurate Chinese information, the existing Chinese art poster generation process must be manually intervened, such as manually designing stylized Chinese characters (such as artistic fonts), and then adding stylized Chinese characters to the corresponding positions of the background image, so as to design a Chinese art poster that meets the requirements. The generation efficiency of Chinese art posters is low. Therefore, how to balance the generation efficiency and generation quality of Chinese art posters has become a major problem in the generation of Chinese art posters. Summary of the invention

[0004] The present invention provides a Chinese art poster generation method, device, computer equipment and storage medium to solve the problem that it is difficult to strike a balance between the generation efficiency and the generation quality of the Chinese art poster in the existing Chinese art poster generation method.

[0005] A method for generating a Chinese art poster is provided, comprising:

[0006] Based on the background image and the Chinese promotional text, Chinese layout information of the Chinese promotional text in the background image is generated, and the background image and the Chinese promotional text are obtained through the poster description information input by the user;

[0007] Based on the Chinese layout information, the background image is positioned and segmented to obtain a local image of the area where the Chinese promotional text is located;

[0008] Based on the Chinese propaganda text and the local image, the pre-trained target prediction model is used to perform text style prediction to obtain the text style information of the Chinese propaganda text;

[0009] Perform style and graphic conversion processing on the Chinese propaganda text based on the text style information to generate the target Chinese graphic;

[0010] Based on the Chinese layout information, the target Chinese graphics are fused into the background image to obtain a Chinese art poster with Chinese graphics.

[0011] Optionally, the target prediction model includes a text encoder, an image encoder, a fusion network and a prediction network. The pre-trained target prediction model is used to perform text style prediction to obtain text style information of the Chinese promotional text, including:

[0012] Performing text encoding on the Chinese propaganda text by using a text encoder to obtain Chinese text features of the Chinese propaganda text;

[0013] By using an image encoder, the background image and the local image are respectively encoded to obtain background image features and local image features;

[0014] The target fusion feature is obtained by fusing Chinese text features, background image features and local image features through a fusion network;

[0015] Based on the target fusion features, the style of the Chinese promotional text is predicted through the prediction network to obtain the text style information of the Chinese promotional text.

[0016] Optionally, the prediction network includes a color prediction head and a font prediction head, and the text style information includes font features and color features. Based on the target fusion features, the prediction network is used to perform style prediction on the Chinese promotional text to obtain text style information of the Chinese promotional text, including:

[0017] Based on the target fusion features, the color prediction head is used to predict the color of the Chinese promotional text to obtain the color features;

[0018] Based on the target fusion features, the font prediction head is used to predict the font of the Chinese promotional text to obtain the font features.

[0019] Optionally, the text style information includes text style information of each Chinese text line in the Chinese promotional text, the target Chinese graphics include Chinese graphics of each Chinese text line, and the Chinese promotional text is subjected to style and graphic conversion processing based on the text style information to generate the target Chinese graphics, including:

[0020] Performing text encoding on each Chinese text line in the Chinese propaganda text to obtain text features of each Chinese text line;

[0021] Perform stroke encoding on each Chinese character in each Chinese text line to obtain stroke features of each Chinese character in each Chinese text line;

[0022] Based on the text style information, text features and stroke features of each Chinese character of each Chinese text line, style and graphic conversion processing is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line.

[0023] Optionally, the text style information includes font features and color features of each Chinese text line, and style and graphic conversion processing is performed on each Chinese text line to obtain Chinese graphics of each Chinese text line, including:

[0024] The font features, text features and stroke features of each Chinese character of each Chinese text line are respectively fused to obtain Chinese fusion features of each Chinese text line;

[0025] The Chinese fusion features of each Chinese text line are respectively input into the preset generation network to perform denoising and graphic text generation processing to obtain the initial graphics of each Chinese text line;

[0026] Based on the color features of each Chinese text line, the initial graphics of each Chinese text line are rendered in color to obtain the Chinese graphics of each Chinese text line.

[0027] Optionally, the Chinese promotional text includes at least one Chinese text line, and based on the Chinese layout information, the target Chinese graphic is merged into the background image to obtain a Chinese art poster with the Chinese graphic, including:

[0028] Determine the Chinese graphics of each Chinese character in each Chinese text line in the target Chinese graphics, and determine the layout information of each Chinese text line in the Chinese layout information;

[0029] Based on the layout information of each Chinese text line, the Chinese graphics of each Chinese character in each Chinese text line are integrated into the background image by means of geometric pasting to obtain a Chinese art poster.

[0030] Optionally, the target prediction model is trained in the following way:

[0031] Acquire multiple Chinese sample images, and pre-process the multiple Chinese sample images respectively to obtain a standard background image of each Chinese sample image, standard style information of standard Chinese, and a standard area image of standard Chinese in the Chinese sample image;

[0032] Input the standard Chinese, standard background image and standard area image of the Chinese sample image into the preset network for style prediction to obtain the predicted style information of the standard Chinese;

[0033] Determine the total loss value based on the predicted style information and the standard style information;

[0034] When the total loss value does not meet the convergence condition, continue to iterate the parameters of the preset network according to multiple Chinese sample graphs until the total loss value meets the convergence condition, determine that the preset network converges, and output the converged preset network as the target prediction model.

[0035] Provided is a Chinese art poster generation device, comprising:

[0036] A layout module, used to generate Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text, wherein the background image and the Chinese promotional text are obtained through poster description information input by the user;

[0037] A segmentation module is used to locate and segment the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located;

[0038] A prediction module is used to perform text style prediction based on the Chinese promotional text and the local image using a pre-trained target prediction model to obtain text style information of the Chinese promotional text;

[0039] A conversion module, used to perform style and graphic conversion processing on the Chinese propaganda text based on the text style information to generate a target Chinese graphic;

[0040] The fusion module is used to fuse the target Chinese graphics into the background image based on the Chinese layout information to obtain a Chinese art poster with Chinese graphics.

[0041] A computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned Chinese art poster generation method are implemented.

[0042] A computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned Chinese art poster generation method are implemented.

[0043] In a solution provided by the above-mentioned Chinese art poster generation method, device, computer equipment and storage medium, based on the background image and Chinese promotional text, Chinese layout information of the Chinese promotional text in the background image is generated, and the background image and the Chinese promotional text are obtained through the poster description information input by the user; based on the Chinese layout information, the background image is positioned and segmented to obtain a local image of the area where the Chinese promotional text is located, and based on the Chinese promotional text and the local image, a pre-trained target prediction model is used to perform text style prediction to obtain text style information of the Chinese promotional text; based on the text style information, style and graphic conversion processing is performed on the Chinese promotional text to generate a target Chinese graphic; based on the Chinese layout information, the target Chinese graphic is fused into the background image to obtain a Chinese art poster with a Chinese graphic. In the present embodiment, text style information is obtained based on prediction of the local image of the background image and the area where the Chinese promotional text is located. Chinese fonts located in different layout positions have different styles, which improves the diversity of text style information. The text style information is then converted into target Chinese graphics, which are then merged to obtain a Chinese art poster. This not only improves the accuracy and style diversity of the fonts in the Chinese art poster, but also makes the Chinese font styles in different areas echo their backgrounds, which can effectively improve the visual effects of the Chinese art posters. High-quality Chinese art posters can be quickly generated without human intervention, which improves the generation efficiency of Chinese art posters while taking into account their generation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0045] Figure 1 is a schematic diagram of an application environment of a method for generating a Chinese art poster in an embodiment of the present invention;

[0046] Figure 2 is a flow chart of a method for generating a Chinese art poster in one embodiment of the present invention;

[0047] Figure 3 This Figure 2 A schematic diagram of an implementation process of step S30;

[0048] Figure 4 This Figure 2 A schematic diagram of an implementation process of step S40;

[0049] Figure 5 is a schematic diagram of stroke coding of a Chinese character in one embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of generating an initial graphic of a Chinese text line in one embodiment of the present invention;

[0051] Figure 7 This Figure 2 A schematic diagram of an implementation process of step S50;

[0052] Figure 8 is a structural schematic diagram of a Chinese art poster generating device in one embodiment of the present invention;

[0053] Fig. 9 It is a structural diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0056] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0058] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0059] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0060] The Chinese art poster generation method provided by the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the Chinese art poster generation method is applied in a Chinese art poster generation system, and the Chinese art poster system includes: Figure 1 The client and server shown communicate with each other through a network, and are used to take into account both the generation efficiency and the generation quality of the Chinese art posters when generating the Chinese art posters.

[0061] When a user needs to generate a Chinese art poster, the user can input the poster description information of the poster to be generated through the client, and the server obtains the poster description information input by the user through the client, and obtains the background image and Chinese promotional text based on the poster description information. For example, the poster description information includes the background image and the Chinese promotional text. Then, the server generates Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text; performs image positioning and segmentation on the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located, and uses the pre-trained target prediction model to perform text style prediction based on the Chinese promotional text and the local image to obtain text style information of the Chinese promotional text; performs style and graphic conversion processing on the Chinese promotional text based on the text style information to generate a target Chinese graphic; and fuses the target Chinese graphic into the background image based on the Chinese layout information to obtain a Chinese art poster with a Chinese graphic. In the present embodiment, text style information is obtained based on prediction of the local image of the background image and the area where the Chinese promotional text is located. Chinese fonts located in different layout positions have different styles, which improves the diversity of text style information. The text style information is then converted into target Chinese graphics, which are then merged to obtain a Chinese art poster. This not only improves the accuracy and style diversity of the fonts in the Chinese art poster, but also makes the Chinese font styles in different areas echo their backgrounds, which can effectively improve the visual effects of the Chinese art posters. High-quality Chinese art posters can be quickly generated without human intervention, which improves the generation efficiency of Chinese art posters while taking into account their generation quality.

[0062] The client, also known as the user end, refers to the program that corresponds to the server and provides local services to customers. The client can be installed on, but not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0063] In one embodiment, if Figure 2 As shown, a method for generating Chinese art posters is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps:

[0064] S10: Based on the background image and the Chinese promotional text, generate Chinese layout information of the Chinese promotional text in the background image.

[0065] After obtaining the background image and the Chinese promotional text, the server generates Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text, that is, Chinese layout information of the Chinese promotional text. The Chinese promotional text includes at least one Chinese text line, and the Chinese layout information includes layout information of each Chinese text line in the Chinese promotional text. The layout information of the Chinese text line can be the geometric center coordinates of the Chinese text line in the background image.

[0066] For example, the background image and Chinese promotional text may be directly input into the layout model, and the layout model may be used to predict the layout position of the Chinese promotional text in the background image to obtain the Chinese layout information of the Chinese promotional text in the background image.

[0067] In other embodiments, the layout model may be a neural network model obtained by training a plurality of Chinese sample images after improving a preset generation network. For example, the layout model may be a neural network model obtained by training a plurality of Chinese sample images after adding a text attribute encoding layer to a traditional generative adversarial network. Adding a text attribute encoding layer to a traditional generative adversarial network and performing deep learning training may improve the accuracy of the layout model, thereby improving the layout effect of the layout model.

[0068] The background image and Chinese promotional text are obtained through the poster description information input by the user. When the user needs to generate a Chinese art poster, the user can input the poster description information of the poster to be generated through the client, and the server obtains the poster description information input by the user through the client, and obtains the background image and Chinese promotional text based on the poster description information.

[0069] For example, the user inputs the background image and Chinese promotional text (i.e., the Chinese text information that needs to appear in the Chinese art poster) of the Chinese art poster to be generated through the client, and the client transmits the background image and Chinese promotional text as poster description information to the server, that is, after the server receives the poster description information, it can directly obtain the background image and Chinese promotional text from the poster description information. That is, the poster description information contains the background image and Chinese promotional text.

[0070] In other embodiments, the poster description information may also include only text information. After the server obtains the poster description information, the server will first use a traditional large language model (such as a GPT model) to identify and classify the poster description information input by the user, and obtain the background-related text and Chinese promotional text in the poster description information. Among them, the background-related text is the description information about the background image input by the user, and the Chinese promotional text is the Chinese text that needs to be displayed in the Chinese art poster. Then, the server uses a text-to-image generation model to generate an image of the background-related text to obtain a background image. Users do not need to upload background images, they only need to upload text information to automatically generate background images, which can cover different scenes and improve user experience.

[0071] S20: performing image positioning and segmentation on the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located.

[0072] After obtaining the Chinese layout information of the Chinese promotion text in the background image, the server performs image positioning and segmentation on the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotion text is located.

[0073] For example, the Chinese promotional text includes at least one Chinese text line, and the Chinese layout information includes the geometric center coordinates of each Chinese text line in the Chinese promotional text. Based on the geometric center coordinates of each Chinese text line, the area where each Chinese text line is located in the background image is determined and segmented to obtain a local image of the area where each Chinese text line is located, that is, a local image of the area where the Chinese promotional text is located is obtained, that is, the local image of the area where the Chinese promotional text is located includes the local image of the area where each Chinese text line is located.

[0074] S30: Based on the Chinese promotional text, the background image and the local image, a pre-trained target prediction model is used to perform text style prediction to obtain text style information of the Chinese promotional text.

[0075] After obtaining the local image of the area where the Chinese promotional text is located, the server uses the pre-trained target prediction model to perform text style prediction based on the Chinese promotional text, background image and local image to obtain the text style information of the Chinese promotional text. The style of the Chinese promotional text is predicted based on the local image in the background image, so that the Chinese fonts in different layout positions in the background image have different styles, which improves the diversity of text style information.

[0076] In other embodiments, the style of the Chinese promotional text can be further predicted based on the background image and the local image, so that the Chinese fonts in different layout positions have different styles, thereby improving the diversity of text style information and making the text style information take into account both the style of the local area and the style of the overall background image, thereby improving the overall harmony of the subsequent Chinese art poster and further improving its visual effect and quality.

[0077] The Chinese promotional text includes at least one Chinese text line, and the text style information of the Chinese promotional text includes the text style information of each Chinese text line in the Chinese promotional text. The text style information of each Chinese character in the same Chinese text line is the same, that is, the text style information of each Chinese text line is also the text style information of each Chinese character in the corresponding Chinese text line. That is, the text style information of the Chinese promotional text includes the text style information of each Chinese character in each Chinese text line.

[0078] S40: Performing style and graphic conversion processing on the Chinese promotional text based on the text style information to generate a target Chinese graphic.

[0079] After obtaining the text style information of the Chinese promotion text, the server performs style and graphic conversion processing on the Chinese promotion text based on the text style information to generate a target Chinese graphic.

[0080] For example, the server performs style conversion on the Chinese promotion text based on the text style information to obtain the stylized Chinese promotion text, and then converts the stylized Chinese promotion text into a graphic format to obtain a target Chinese graphic. The target Chinese graphic includes the graphic of each Chinese character in each Chinese text line.

[0081] S50: Based on the Chinese layout information, the target Chinese graphics are integrated into the background image to obtain a Chinese art poster with Chinese graphics.

[0082] After obtaining the target Chinese graphics, the target Chinese graphics are merged into the background image based on the Chinese layout information to obtain a Chinese art poster with Chinese graphics. Graphical Chinese fonts can improve the accuracy of generated Chinese characters, thereby improving the generation quality of Chinese art posters.

[0083] For example, the Chinese promotional text includes at least one Chinese text line, the Chinese layout information includes the layout information of all Chinese text lines, and the layout information of each Chinese text line includes the coordinate position of each Chinese character in the Chinese text line in the poster background image. The target Chinese graphics include the graphics of each Chinese character in each Chinese text line. According to the coordinate position of each Chinese character in each Chinese text line in the poster background image, the graphics of the corresponding Chinese characters can be pasted to the corresponding position of the poster background image, and then solidified and fused to obtain a Chinese art poster with Chinese graphics.

[0084] In the present embodiment, text style information is obtained based on prediction of the local image of the background image and the area where the Chinese promotional text is located. Chinese fonts located in different layout positions have different styles, which improves the diversity of text style information. The text style information is then converted into target Chinese graphics, which are then merged to obtain a Chinese art poster. This not only improves the accuracy and style diversity of the fonts in the Chinese art poster, but also makes the Chinese font styles in different areas echo their backgrounds, which can effectively improve the visual effects of the Chinese art posters. High-quality Chinese art posters can be quickly generated without human intervention, which improves the generation efficiency of Chinese art posters while taking into account their generation quality.

[0085] In one embodiment, the target prediction model includes a text encoder, an image encoder, a fusion network, and a prediction network. Figure 3 As shown, in step S30, the text style prediction is performed using the pre-trained target prediction model to obtain the text style information of the Chinese promotional text, which specifically includes the following steps:

[0086] S31: Encode the Chinese promotional text through a text encoder to obtain Chinese text features of the Chinese promotional text.

[0087] The Chinese promotional text, the background image and the local image are input into the target prediction model, and each Chinese text line in the Chinese promotional text is encoded by a text encoder to obtain Chinese text features of the Chinese promotional text. The Chinese text features of the Chinese promotional text include text features of each Chinese text line.

[0088] S32: performing image encoding on the background image and the local image respectively through an image encoder to obtain background image features and local image features.

[0089] At the same time, the background image and the local image are respectively encoded by an image encoder to obtain background image features of the background image and local image features of the local image.

[0090] S33: The Chinese text features, background image features and local image features are fused through a fusion network to obtain target fusion features.

[0091] Then, the text features of each Chinese text line in the Chinese text features, the background image features and the local image features are input into the fusion network for feature fusion to obtain the target fusion features. The target fusion features include the fusion features of each Chinese text line.

[0092] The target fusion feature can be directly obtained by splicing the Chinese text feature, the background image feature and the local image feature. In other embodiments, the Chinese text feature, the background image feature and the local image feature can also be fused based on the attention mechanism to obtain a more accurate target fusion feature.

[0093] Among them, the fusion network includes two two-layer fully connected neural networks. The intermediate mapping dimension of the fusion network is 1024, and finally a 512-dimensional fusion vector is output, that is, the target fusion feature is 512-dimensional.

[0094] S34: Based on the target fusion features, the style of the Chinese promotional text is predicted through the prediction network to obtain the text style information of the Chinese promotional text.

[0095] Finally, the fusion features of each Chinese text line in the target fusion features are input into the prediction network, and the prediction network performs style prediction on each Chinese text line of the Chinese promotional text based on the target fusion features to obtain the text style information of the Chinese promotional text. The text style information of the Chinese promotional text includes the text style information of each Chinese text line in the Chinese promotional text.

[0096] In other embodiments, the local image can be encoded by an image encoder to obtain local image features, and then the Chinese text features and the local image features can be fused by a fusion network to obtain target fusion features. Based on the target fusion features, the style of the Chinese promotional text can be predicted by a prediction network to obtain text style information of the Chinese promotional text.

[0097] In this embodiment, the specific structure of the target prediction model is clarified; the Chinese promotional text is encoded by a text encoder to obtain Chinese text features of the Chinese promotional text; the background image and the local image are respectively encoded by an image encoder to obtain background image features and local image features; the Chinese text features, background image features and local image features are fused by a fusion network to obtain target fusion features; based on the target fusion features, the style of the Chinese promotional text is predicted by a prediction network to obtain text style information of the Chinese promotional text, and the process of predicting the text style information of the Chinese promotional text by the target prediction model is clarified, the efficiency of text style information prediction is improved, and the accuracy of text style information is improved, so that different Chinese texts have different styles, while taking into account the styles of the local area and the overall background image, echoing the overall background style, improving the overall harmony of the subsequent Chinese art posters, and improving the visual effects and quality of the Chinese art posters.

[0098] In one embodiment, the prediction network includes a color prediction head and a font prediction head, and the text style information includes font features and color features. In step S34, based on the target fusion features, the prediction network is used to perform style prediction on the Chinese promotional text to obtain the text style information of the Chinese promotional text, which specifically includes the following steps:

[0099] S341: Based on the target fusion features, the color prediction head is used to predict the color of the Chinese promotional text to obtain color features.

[0100] In this embodiment, the prediction network includes a color prediction head and a font prediction head. After obtaining the target fusion feature, the fusion features of each Chinese text line in the target fusion feature are respectively input into the prediction network, and the color prediction head is used to perform color prediction on each Chinese text line in the Chinese promotional text to obtain the color features of each Chinese text line in the Chinese promotional text.

[0101] Specifically, the color prediction head is used to predict the color of the Chinese promotional text to obtain color features, including: using the first activation function to activate the fusion features of each Chinese text line in the target fusion feature to obtain the first activation feature of each Chinese text line; normalizing the first activation feature of each Chinese text line to obtain the normalized feature of each Chinese text line; mapping the normalized feature of each Chinese text line to the RGB color space for color prediction to obtain the 3D color feature of each Chinese text line. By first activating and normalizing the target fusion feature, the nonlinear threshold of the target fusion feature is automatically adjusted to facilitate subsequent color mapping, thereby quickly obtaining accurate color features.

[0102] The first activation function and the second activation function can both be GELU (Gaussian Error Linear Unit) activation functions, that is, Gaussian error linear activation functions. Compared with the traditional ReLU activation function, the use of GELU activation function activation makes the first activation feature have better smoothness. Among them, the sigmoid function can be used to normalize the first activation feature, which has faster compression speed and more stable features.

[0103] S342: Based on the target fusion features, the font prediction head is used to predict the font of the Chinese promotional text to obtain font features.

[0104] After the target fusion features are input into the prediction network, the font prediction head is used to perform font prediction on each Chinese text line in the Chinese promotional text to obtain the font features of each Chinese text line.

[0105] Specifically, the font prediction head is used to predict the font of the Chinese promotional text to obtain (each Chinese font feature, including: using the second activation function to activate the fusion feature of each Chinese text line in the target fusion feature to obtain the second activation feature of each Chinese text line; mapping the second activation feature of each Chinese text line to the preset font style space for font prediction to obtain the font feature. The preset font style space is pre-entered with all font styles existing in the market, which is convenient for subsequent font prediction.

[0106] The second activation function and the first activation function may be the same. The first activation function and the second activation function may both be GELU activation functions. In other embodiments, the second activation function may also be a sigmoid function or a ReLU function.

[0107] In this embodiment, the prediction network includes a color prediction head and a font prediction head. Based on the target fusion feature, the color prediction head performs color prediction on the Chinese promotional text to obtain color features, and based on the target fusion feature, the font prediction head performs font prediction on the Chinese promotional text to obtain font features. The specific acquisition process of the text style information of the Chinese promotional text is clarified, the information diversity of the text style information is improved, and thus the style diversity of the target Chinese graphics is improved.

[0108] In one embodiment, the text style information includes the text style information of each Chinese character in the Chinese promotion text, and the target Chinese graphics include the Chinese graphics of each Chinese text line in the Chinese promotion text. Figure 4 As shown, in step S40, the Chinese promotional text is subjected to style and graphic conversion processing based on the text style information to generate a target Chinese graphic, which specifically includes the following steps:

[0109] S41: Perform text encoding on each Chinese text line in the Chinese promotional text to obtain the text features of each Chinese text line.

[0110] The Chinese promotional text includes at least one Chinese text line, and the target Chinese graphics of the Chinese promotional text include the Chinese graphics of each Chinese text line. The Chinese graphics of a Chinese text line include the Chinese graphics of each Chinese character in that Chinese text line.

[0111] When generating the target Chinese graphics, first perform text encoding on each Chinese text line in the Chinese promotional text to obtain the text features of each Chinese text line. The text features can be semantic features or the graphic features of that Chinese text line (including spatial relationship features, shape features, etc.).

[0112] S42: Perform stroke encoding on each Chinese character in each Chinese text line to obtain the stroke features of each Chinese character in each Chinese text line.

[0113] Then, perform stroke encoding on each Chinese character in each Chinese text line in the Chinese promotional text to obtain the stroke features of each Chinese character in each Chinese text line. Split and count the strokes of the Chinese font, and 32 standard strokes can be counted, that is, there are 32 types of standard strokes in the Chinese font; then encode the Chinese characters according to these 32 standard strokes to obtain a 32-dimensional stroke feature. Among them, the 32-dimensional stroke feature includes the quantity of each of the 32 standard strokes.

[0114] For example, taking a certain Chinese character '通' as an example, as Figure 5 shown, the standard strokes of the character '通' include dot (丶), horizontal (一), vertical (丨), right-falling stroke ( ), horizontal hook (乛), horizontal fold hook ( ), horizontal fold and flick ( ) for a total of 7 standard strokes; among them, the quantities of the standard strokes 丶, 一, 丨 are all 2, and the quantities of the standard strokes , 乛, and are all 1, and the quantities of the remaining 25 standard strokes are all 0. Thus, the stroke feature of the character '通' is: 2220100000000110000001000000000.

[0115] S43: Based on the text style information, text features of each Chinese text line, and the stroke features of each Chinese character, perform style and graphic conversion processing on each Chinese text line to obtain the Chinese graphics of each Chinese text line.

[0116] Finally, based on the text style information, text features, and stroke features of each Chinese text line, each Chinese text line is subjected to style and graphic conversion processing to obtain Chinese graphics of each Chinese text line. The Chinese graphics include Chinese graphics of each Chinese character in the Chinese text line, which can ensure the accuracy of each Chinese character in the output Chinese text line and improve the accuracy of Chinese generation.

[0117] For example, the text style information, text features and stroke features of each Chinese text line and each Chinese character can be input into a pre-trained font generation model to perform font style transfer generation to obtain stylized Chinese text lines, and then the stylized Chinese text lines are converted into a graphic format to obtain Chinese graphics of each Chinese character.

[0118] In this embodiment, text encoding is performed on each Chinese text line in the Chinese promotional text to obtain the text features of each Chinese text line, and then stroke encoding is performed on each Chinese character in each Chinese text line to obtain the stroke features of each Chinese character in each Chinese text line. Then, based on the text style information, text features and stroke features of each Chinese character of each Chinese text line, style and graphic conversion processing is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line, which can ensure the accuracy of the output Chinese characters and improve the accuracy of Chinese generation.

[0119] In one embodiment, the text style information of each Chinese character includes the font features and color features of each Chinese character. In step S43, based on the text style information, text features and stroke features of each Chinese character of each Chinese text line, style and graphic conversion processing is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line, which specifically includes the following steps:

[0120] S431: Fusing the font features, text features, and stroke features of each Chinese character of each Chinese text line to obtain Chinese fusion features of each Chinese text line.

[0121] After obtaining the text style information, text features and stroke features of each Chinese character of each Chinese text line, the font features, text features and stroke features of each Chinese character of the Chinese text line are fused. For example, the text style information, text features and stroke features of each Chinese character can be directly concatenated to obtain the Chinese fusion features of the Chinese text line; by traversing all Chinese text lines, the Chinese fusion features of each Chinese text line can be obtained.

[0122] S432: Inputting the Chinese fusion features of each Chinese text line into a preset generation network respectively, performing denoising and graphic text generation processing, and obtaining an initial graphic of each Chinese text line.

[0123] Then, the Chinese fusion features of the Chinese text line are input into the preset generation network, and after random noise denoising, the graphic text generation process is performed to obtain the initial graphic of the Chinese text line; by traversing all Chinese text lines, the initial graphics of each Chinese text line can be obtained.

[0124] The preset generation network may be a convolutional neural network Unet. Figure 6 As shown, the Chinese text line is: Big Fish and Begonia. The server can splice the font features (Predicted Font Features), text features (Text Embedding) and stroke features (Strokes Embedding) of the Chinese text line, and input the spliced ​​Chinese fusion features into the Unet network for graphic diffusion generation. Random noise (Random Noise) is input into the Unet network diffusion process (DiffusionProcess), and then denoising and artistic font graphics are generated. The final generated font (GeneratedFont) is a stylized Big Fish and Begonia.

[0125] The UNet network is a U-shaped network structure that can effectively capture multi-scale feature information and is suitable for fields such as semantic segmentation and image processing. In other embodiments, the preset generation network can also be other types of generative neural networks, which will not be described in detail here.

[0126] S433: Based on the color features of each Chinese text line, color rendering is performed on the initial graphics of each Chinese text line to obtain the Chinese graphics of each Chinese text line.

[0127] After obtaining the initial graphics of each Chinese text line, the initial graphics of each Chinese text line are color rendered based on the color features of each Chinese text line to obtain the Chinese graphics of each Chinese text line. For example, the color features can be converted into corresponding color information, and then the initial graphics of each Chinese text line are color rendered based on the color information to obtain the Chinese graphics in each Chinese text line.

[0128] In other embodiments, the Chinese layout information includes the coordinate position of each Chinese character in each Chinese text line; after obtaining the text style information and text features of each Chinese text line, the text style information and text features of the Chinese text line can be used as the text style information and text features of each Chinese character in the Chinese text line. Then, the font features, text features and stroke features of the Chinese characters are fused, such as directly splicing the text style information, text features and stroke features of the Chinese characters to obtain the Chinese fusion features of the Chinese characters; all Chinese characters in all Chinese text lines are traversed to obtain the Chinese fusion features of each Chinese character in each Chinese text line. Then, the Chinese fusion features of the Chinese characters are input into the preset generation network, and denoising and graphic text generation processing are performed to obtain the initial graphics of the Chinese characters; all Chinese characters in all Chinese text lines are traversed to obtain the initial graphics of each Chinese character in each Chinese text line. Finally, based on the color features of each Chinese character, the initial graphics of each Chinese character are rendered in color to obtain the Chinese graphics of each Chinese character. It is convenient to paste the Chinese graphics of each Chinese character directly onto the poster background image based on the coordinate position of each Chinese character to obtain a Chinese art poster, which is simple and convenient.

[0129] In this embodiment, the font features, text features and stroke features of each Chinese text line are fused to obtain the Chinese fusion features of each Chinese text line, and then the Chinese fusion features of each Chinese text line are input into the preset generation network to perform denoising and graphic text generation processing to obtain the initial graphics of each Chinese text line, and finally, based on the color features of each Chinese text line, the initial graphics of each Chinese text line are rendered in color to obtain the Chinese graphics of each Chinese text line, and the specific acquisition process of the Chinese graphics of each Chinese text line is refined. First, the font features, text features and stroke features are fused and denoised to improve the accuracy of the font, and then color rendering is performed to obtain a graphic Chinese font with more accurate color, which improves the accuracy and style diversity of Chinese font generation.

[0130] In one embodiment, the Chinese promotional text includes at least one Chinese text line. Figure 7 As shown, in step S50, based on the Chinese layout information, the target Chinese graphic is merged into the background image to obtain a Chinese art poster with Chinese graphics, which specifically includes the following steps:

[0131] S51: Determine the Chinese graphics of each Chinese text line in the target Chinese graphics, and determine the layout information of each Chinese text line in the Chinese layout information.

[0132] S52: Based on the layout information of each Chinese text line, the Chinese graphics of each Chinese text line are integrated into the background image by using a geometric pasting method to obtain a Chinese art poster.

[0133] In this embodiment, the Chinese layout information includes layout information of each Chinese text line. The target Chinese graphics include Chinese graphics of each Chinese text line. After obtaining the Chinese layout information and the target Chinese graphics, the Chinese graphics of each Chinese text line are determined in the target Chinese graphics, and the layout information of each Chinese text line is determined in the Chinese layout information. Then, based on the layout information of each Chinese text line, the Chinese graphics of each Chinese text line are integrated into the background image by geometric pasting to obtain a Chinese art poster.

[0134] In other embodiments, the Chinese layout information includes the coordinate position of each Chinese character in each Chinese text line, and the target Chinese graphics include the Chinese graphics of each Chinese character in each Chinese text line. Based on the coordinate position of each Chinese character, the Chinese graphics of each Chinese character can be directly pasted to the background image and solidified to obtain a Chinese art poster.

[0135] In this embodiment, by determining the Chinese graphics of each Chinese text line in the target Chinese graphics, and determining the layout information of each Chinese text line in the Chinese layout information, and then based on the layout information of each Chinese text line, the Chinese graphics in each Chinese text line are integrated into the background image by geometric pasting to obtain a Chinese art poster, and the specific process of integrating various types of information to obtain a Chinese art poster is clarified. The Chinese graphics of each Chinese text line are directly pasted to the corresponding position by pasting, and the operation is simple and convenient.

[0136] In one embodiment, the Chinese promotional text includes at least one Chinese text line. In step S10, based on the background image and the Chinese promotional text, Chinese layout information of the Chinese promotional text in the background image is generated, which specifically includes the following steps:

[0137] S11: Determine the number of text lines in the Chinese promotional text and the length-to-width ratio of each Chinese text line;

[0138] Among them, the number of text lines represents the total number of all Chinese text lines in the Chinese promotional text; the length-to-width ratio of a Chinese text line represents the ratio of the width to the length of the Chinese text line.

[0139] S12: Encode the number of text lines and the aspect ratio respectively to obtain a text line number vector and a text line ratio vector, and encode the background image to obtain a background vector.

[0140] The number of text lines can be encoded through a quantity encoder to obtain a text line number vector; the length and width ratio of each Chinese text line can be encoded using a ratio to obtain a text line ratio vector of each Chinese text line; the background image can be encoded using an image encoder of a generative adversarial network to obtain a background vector.

[0141] S13: Input the background vector, the text line number vector and the text line ratio vector into the image generation network to generate an image and obtain an initial generated image.

[0142] For example, the image generation network can be a generative adversarial network, and the background vector, the text line number vector, and the text line ratio vector are input into the generative adversarial network for image generation to obtain an initial generated image. Using multi-dimensional vectors such as the background vector, the text line number vector, and the text line ratio vector for image generation can improve the quality of the initial generated image, thereby improving the Chinese layout effect of the initial generated image.

[0143] S14: Using a text recognition model to recognize and locate the Chinese characters in the initially generated image, and obtaining Chinese layout information of the Chinese promotional text.

[0144] Finally, a text recognition model is used to recognize and locate the Chinese characters in the initial generated image, and the Chinese layout information of the Chinese promotional text is obtained. The text recognition model can be an OCR (Optical Character Recognition) model, which is fast and accurate.

[0145] In this embodiment, by determining the number of text lines in the Chinese promotional text and the aspect ratio of each Chinese text line, and encoding the number of text lines and the aspect ratio respectively, a text line number vector and a text line ratio vector are obtained, and the background image is encoded to obtain a background vector, and then the background vector, the text line number vector and the text line ratio vector are input into the image generation network for image generation to obtain an initial generated image, and finally a text recognition model is used to identify and locate the Chinese in the initial generated image to quickly obtain accurate Chinese layout information.

[0146] In one embodiment, before step S30, that is, before using the pre-trained target prediction model to perform text style prediction, it is necessary to first obtain the target prediction model, and then execute step S30, that is, based on the Chinese promotional text, background image and local image, use the pre-trained target prediction model to perform text style prediction to obtain the text style information of the Chinese promotional text.

[0147] Among them, the target prediction model is trained in the following way:

[0148] S01: Acquire multiple Chinese sample images, and pre-process the multiple Chinese sample images respectively to obtain a standard background image of each Chinese sample image, standard style information of standard Chinese, and a standard area image of standard Chinese in the Chinese sample image.

[0149] The server first obtains multiple Chinese sample images, which are historical Chinese posters from various industries that have appeared in the past. Then, the server needs to pre-process the multiple Chinese sample images separately to obtain the standard background image of each Chinese sample image, the standard style information of the standard Chinese, and the standard area image of the standard Chinese in the Chinese sample image. Among them, the standard Chinese in the Chinese sample image is the Chinese information that appears in the standard Chinese; the standard background image is the image after all the text information on the Chinese sample image is removed; the standard style information of the standard Chinese includes at least the standard font and standard color, and in other embodiments, it may also include size, etc.; the standard area image of the standard Chinese in the Chinese sample image refers to the local image of the area where the standard Chinese is located in the Chinese sample image.

[0150] Then, the server uses the standard background images and standard area images of the plurality of Chinese sample images as training data.

[0151] S02: Inputting the standard Chinese, standard background image and standard area image of the Chinese sample image into a preset network for style prediction to obtain predicted style information of the standard Chinese.

[0152] The server inputs the standard background image and the standard area image of the Chinese sample image into a preset network for style prediction to obtain the predicted style information of the standard Chinese.

[0153] Specifically, the preset network includes a text encoder, an image encoder, a fusion network and a prediction network. After the standard background image and the standard region image of the Chinese sample image are input into the preset network, the following steps are implemented: the standard Chinese is encoded by the text encoder to obtain the standard text features of the standard Chinese; the standard background image and the standard region image are respectively encoded by the image encoder to obtain the overall features of the standard background image and the local features of the standard region image; the standard text features, the overall features and the local features are fused by the fusion network to obtain the standard fusion features; based on the standard fusion features, the style of the standard Chinese is predicted by the prediction network to obtain the predicted style information of the standard Chinese.

[0154] The prediction network includes a color prediction head and a font prediction head. The standard fusion features are input into the prediction network, and the color prediction head performs color prediction based on the standard fusion features to obtain the predicted color, and the font prediction head performs font prediction based on the standard fusion features to obtain the predicted font. The predicted style information includes the predicted color and the predicted font.

[0155] Among them, the specific processing process of each network structure in the preset network can refer to the generation process of text style information in the previous article, which will not be repeated here.

[0156] S03: Determine a total loss value based on the predicted style information and the standard style information.

[0157] After obtaining the predicted style information of standard Chinese, the total loss value is determined according to the predicted style information of standard Chinese and the standard style information.

[0158] The standard style information of standard Chinese includes at least a standard font and a standard color; the predicted style information includes a predicted color and a predicted font. The total loss value is determined according to the predicted style information and the standard style information, including: determining a font loss value according to the standard font and the predicted font, determining a color loss value according to the standard color and the predicted color, and then determining a total loss value based on the font loss value and the color loss value.

[0159] Among them, the total loss value can be calculated by the following formula:

[0160] ;

[0161] ;

[0162] ;

[0163] in, Represents the total loss value. Represents the color loss value. The loss function of the color prediction head in the preset network is the mean square error loss function. represents the predicted color, Standard colors that represent standard style information, i.e. Represents the mean square error between the predicted color and the standard color. Represents the font loss value. The loss function of the font prediction head in the default network is the Kullback-Leibler (KL) divergence function, that is, represents the KL divergence between the predicted font and the standard font. represents the KL divergence function, Indicates the predicted font, Indicates standard fonts. represents the softmax function. t represents the temperature factor scaling the feature distribution. , t are both preset constant parameters; and It is a model iteration parameter that changes dynamically as training progresses.

[0164] By determining the total loss value in the above manner, the loss of color and font is taken into account, so that the total loss value of the model is more accurate. In other embodiments, the total loss value can also be other, which will not be repeated here.

[0165] S04: When the total loss value does not meet the convergence condition, continue to iterate the parameters of the preset network according to multiple Chinese sample graphs until the total loss value meets the convergence condition, determine that the preset network converges, and output the converged preset network as the target prediction model.

[0166] When the total loss value does not meet the convergence condition, continue to execute steps S02-S04, that is, continue to iterate the parameters of the preset network according to multiple Chinese sample graphs until the total loss value meets the convergence condition, determine that the preset network converges, and output the converged preset network as the target prediction model.

[0167] In this embodiment, the specific training process of the target prediction model is clarified. By training a target prediction model that meets the requirements through standard background images and standard area images of multiple Chinese sample images, the accuracy of the target prediction model can be improved. The subsequent prediction of text style information by the target prediction model can improve the accuracy and prediction efficiency of the text style information, thereby improving the quality and generation efficiency of Chinese art posters.

[0168] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0169] In one embodiment, a Chinese art poster generation device is provided, and the Chinese art poster generation device corresponds to the Chinese art poster generation method in the above embodiment. Figure 8 As shown, the Chinese art poster generation device includes a layout module 801, a segmentation module 802, a prediction module 803, a conversion module 804 and a fusion module 805. The detailed description of each functional module is as follows:

[0170] The layout module 801 is used to generate Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text, where the background image and the Chinese promotional text are obtained through poster description information input by the user;

[0171] The segmentation module 802 is used to locate and segment the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located;

[0172] Prediction module 803, used to perform text style prediction based on the Chinese promotion text and the local image using a pre-trained target prediction model to obtain text style information of the Chinese promotion text;

[0173] The conversion module 804 is used to perform style and graphic conversion processing on the Chinese promotional text based on the text style information to generate a target Chinese graphic;

[0174] The fusion module 805 is used to fuse the target Chinese graphics into the background image based on the Chinese layout information to obtain the target Chinese art poster with the Chinese graphics.

[0175] Optionally, the text style information includes text style information of each Chinese text line in the Chinese promotional text, the target Chinese graphics include Chinese graphics of each Chinese text line, and the conversion module 804 is specifically used for:

[0176] Performing text encoding on each Chinese text line in the Chinese propaganda text to obtain text features of each Chinese text line;

[0177] Perform stroke encoding on each Chinese character in each Chinese text line to obtain stroke features of each Chinese character in each Chinese text line;

[0178] Based on the text style information, text features and stroke features of each Chinese character of each Chinese text line, style and graphic conversion processing is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line.

[0179] Optionally, the text style information includes font features and color features of each Chinese text line, and the conversion module 804 is further configured to:

[0180] The font features, text features and stroke features of each Chinese character of each Chinese text line are respectively fused to obtain Chinese fusion features of each Chinese text line;

[0181] The Chinese fusion features of each Chinese text line are respectively input into the preset generation network to perform denoising and graphic text generation processing to obtain the initial graphics of each Chinese text line;

[0182] Based on the color features of each Chinese text line, the initial graphics of each Chinese text line are rendered in color to obtain the Chinese graphics of each Chinese text line.

[0183] Optionally, the Chinese promotion text includes at least one Chinese text line, and the fusion module 805 is specifically used to:

[0184] Determine the Chinese graphics of each Chinese character in each Chinese text line in the target Chinese graphics, and determine the layout information of each Chinese text line in the Chinese layout information;

[0185] Based on the layout information of each Chinese text line, the Chinese graphics of each Chinese character in each Chinese text line are integrated into the background image by means of geometric pasting to obtain a Chinese art poster.

[0186] Optionally, the Chinese art poster generation device further includes a training module 806, and the training module 806 is used to train and obtain a target prediction model in the following manner:

[0187] Acquire multiple Chinese sample images, and pre-process the multiple Chinese sample images respectively to obtain a standard background image of each Chinese sample image, standard style information of standard Chinese, and a standard area image of standard Chinese in the Chinese sample image;

[0188] Input the standard Chinese, standard background image and standard area image of the Chinese sample image into the preset network for style prediction to obtain the predicted style information of the standard Chinese;

[0189] Determine the total loss value based on the predicted style information and the standard style information;

[0190] When the total loss value does not meet the convergence condition, continue to iterate the parameters of the preset network according to multiple Chinese sample graphs until the total loss value meets the convergence condition, determine that the preset network converges, and output the converged preset network as the target prediction model.

[0191] The specific definition of the Chinese art poster generation device can be found in the definition of the Chinese art poster generation method mentioned above, which will not be repeated here. Each module in the above-mentioned Chinese art poster generation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0192] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used and generated by the Chinese art poster generation method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a Chinese art poster generation method is implemented.

[0193] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned Chinese art poster generation method are implemented.

[0194] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned Chinese art poster generation method are implemented.

[0195] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0196] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0197] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for generating Chinese art posters, characterized in that: include: Based on the background image and the Chinese promotional text, generating Chinese layout information of the Chinese promotional text in the background image, wherein the background image and the Chinese promotional text are obtained through poster description information input by a user; Performing image positioning and segmentation on the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located; Based on the Chinese promotional text and the partial image, a pre-trained target prediction model is used to perform text style prediction to obtain text style information of the Chinese promotional text; Performing style and graphic conversion processing on the Chinese promotional text based on the text style information to generate a target Chinese graphic; Based on the Chinese layout information, the target Chinese graphic is merged into the background image to obtain a target Chinese art poster having the Chinese graphic; The Chinese promotional text includes at least one Chinese text line, and the step of generating Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text includes: Determine the number of text lines in the Chinese promotional text and the aspect ratio of each Chinese text line, wherein the number of text lines represents the total number of all Chinese text lines in the Chinese promotional text; the aspect ratio of the Chinese text line represents the ratio of the width to the length of the Chinese text line; Encoding the number of text lines and the aspect ratio respectively to obtain a text line number vector and a text line ratio vector, and encoding the background image to obtain a background vector; Inputting the background vector, the text line quantity vector and the text line ratio vector into an image generation network to generate an image, thereby obtaining an initial generated image; A text recognition model is used to recognize and locate the Chinese characters in the initially generated image, so as to obtain Chinese layout information of the Chinese promotional text.

2. The Chinese art poster generation method according to claim 1, characterized in that: The text style information includes text style information of each Chinese text line in the Chinese promotion text, the target Chinese graphics include Chinese graphics of each Chinese text line, and the style and graphic conversion processing of the Chinese promotion text based on the text style information to generate the target Chinese graphics includes: Performing text encoding on each of the Chinese text lines in the Chinese promotional text to obtain text features of each of the Chinese text lines; Performing stroke encoding on each Chinese character in each Chinese text line to obtain stroke features of each Chinese character in each Chinese text line; Based on the text style information of each Chinese text line, the text features and the stroke features of each Chinese character, style and graphic conversion processing is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line.

3. The method for generating Chinese art posters as claimed in claim 2, characterized in that: The text style information includes the font features and color features of each Chinese text line, and the style and graphic conversion process is performed on each Chinese text line to obtain the Chinese graphics of each Chinese text line, including: Respectively fusing the font features of each Chinese text line, the text features and the stroke features of each Chinese character to obtain Chinese fusion features of each Chinese text line; Inputting the Chinese fusion features of each Chinese text line into a preset generation network respectively, performing denoising and graphic text generation processing, and obtaining an initial graphic of each Chinese text line; Based on the color features of each of the Chinese text lines, color rendering is performed on the initial graphics of each of the Chinese text lines to obtain the Chinese graphics of each of the Chinese text lines.

4. The method for generating Chinese art posters as claimed in claim 1, characterized in that: The target prediction model includes a text encoder, an image encoder, a fusion network and a prediction network. The target prediction model obtained by pre-training is used to perform text style prediction to obtain text style information of the Chinese propaganda text, including: Performing text encoding on the Chinese promotional text by the text encoder to obtain Chinese text features of the Chinese promotional text; By means of the image encoder, the background image and the local image are respectively encoded to obtain background image features and local image features; fusing the Chinese text features, the background image features and the local image features through the fusion network to obtain a target fusion feature; Based on the target fusion feature, the prediction network is used to perform style prediction on the Chinese promotional text to obtain text style information of the Chinese promotional text.

5. The method for generating Chinese art posters as claimed in claim 4, characterized in that: The prediction network includes a color prediction head and a font prediction head, the text style information includes font features and color features, and based on the target fusion features, the prediction network is used to perform style prediction on the Chinese promotional text to obtain the text style information of the Chinese promotional text, including: Based on the target fusion feature, the color prediction head is used to predict the color of the Chinese promotional text to obtain the color feature; Based on the target fusion feature, the font prediction head is used to perform font prediction on the Chinese promotional text to obtain the font feature.

6. The method for generating Chinese art posters as claimed in claim 1, characterized in that: The Chinese promotional text includes at least one Chinese text line, and based on the Chinese layout information, the target Chinese graphic is merged into the background image to obtain a target Chinese art poster having the Chinese graphic, including: Determining the Chinese graphics of each Chinese character in each Chinese text line in the target Chinese graphics, and determining the layout information of each Chinese text line in the Chinese layout information; Based on the layout information of each of the Chinese text lines, the Chinese graphics of each Chinese character in each of the Chinese text lines are integrated into the background image by means of geometric pasting to obtain the target Chinese art poster.

7. The method for generating a Chinese art poster according to any one of claims 1 to 6, characterized in that: The target prediction model is trained in the following way: Acquire multiple Chinese sample images, and pre-process the multiple Chinese sample images respectively to obtain a standard background image of each Chinese sample image, standard style information of standard Chinese, and a standard area image of the standard Chinese in the Chinese sample image; Inputting the standard Chinese, the standard background image and the standard area image of the Chinese sample image into a preset network for style prediction to obtain predicted style information of the standard Chinese; Determining a total loss value according to the predicted style information and the standard style information; When the total loss value does not meet the convergence condition, continue to iterate the parameters of the preset network according to the multiple Chinese sample graphs until the total loss value meets the convergence condition, determine that the preset network converges, and output the converged preset network as the target prediction model.

8. A Chinese art poster generation device, characterized in that: include: A layout module, for generating Chinese layout information of the Chinese promotional text in the background image based on a background image and Chinese promotional text, wherein the background image and the Chinese promotional text are obtained through poster description information input by a user; wherein the Chinese promotional text includes at least one Chinese text line, and the generating Chinese layout information of the Chinese promotional text in the background image based on the background image and the Chinese promotional text comprises: Determine the number of text lines in the Chinese promotional text and the aspect ratio of each Chinese text line, wherein the number of text lines represents the total number of all Chinese text lines in the Chinese promotional text; the aspect ratio of the Chinese text line represents the ratio of the width to the length of the Chinese text line; Encoding the number of text lines and the aspect ratio respectively to obtain a text line number vector and a text line ratio vector, and encoding the background image to obtain a background vector; Inputting the background vector, the text line quantity vector and the text line ratio vector into an image generation network to generate an image, thereby obtaining an initial generated image; Using a text recognition model to recognize and locate Chinese characters in the initially generated image, and obtaining Chinese layout information of the Chinese promotional text; A segmentation module, used to locate and segment the background image based on the Chinese layout information to obtain a local image of the area where the Chinese promotional text is located; A prediction module, configured to perform text style prediction based on the Chinese promotional text and the partial image using a pre-trained target prediction model to obtain text style information of the Chinese promotional text; A conversion module, used for performing style and graphic conversion processing on the Chinese promotional text based on the text style information to generate a target Chinese graphic; A fusion module is used to fuse the target Chinese graphics into the background image based on the Chinese layout information to obtain a target Chinese art poster with the Chinese graphics.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for generating a Chinese art poster as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for generating a Chinese art poster as described in any one of claims 1 to 7 are implemented.

Images