Image generation method, device and storage medium

Abstract

The embodiment of the present invention discloses an image generation method, device and storage medium. The embodiment of the present invention can obtain an input text item, screen out a target template from preset candidate text templates according to the text item; typeset the text item according to the target template to generate a text block; obtain a background image, extract a saliency map of the background image; obtain pixel information of the saliency map, determine the target position of the text block in the background image according to the pixel information of the saliency map; typeset the background image and the text block according to the target position to generate a target image. Therefore, the scheme improves the overall effect of the image while improving the degree of automation of graphic image production.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an image generation method, device and storage medium. Background Art

[0002] Posters with pictures and texts are a very frequently used means of publicity in daily life. They can quickly attract the audience's attention and highlight the publicity information. Since it takes a lot of time and effort to manually design posters, many businesses have begun to try to use design software to make posters.

[0003] In the process of studying and practicing the prior art, the inventor of the present invention found that the current design software is essentially a user inputting the text to be displayed and selecting the template to be used from the template library provided by the design software. Then, the design software fills the text input by the user into the selected template and outputs the prepared poster image. If the user is not satisfied with the layout of the text in the template, the user needs to manually adjust the text based on the design experience, which consumes a lot of energy and time. It can be seen that the current automation level of the production of graphic images such as posters is relatively poor. Summary of the invention

[0004] The embodiments of the present invention provide an image generation method, device and storage medium, aiming to solve the technical problem of low automation level in the production of graphic images.

[0005] An embodiment of the present invention provides an image generation method, including:

[0006] Get the input text item, and filter out the target template from the preset candidate text templates according to the text item:

[0007] Typesetting the text item according to the target template to generate a text section;

[0008] Acquire a background image, and extract a saliency map of the background image;

[0009] Acquiring pixel information of the saliency map, and determining a target position of the text block in the background image according to the pixel information of the saliency map;

[0010] According to the target position, the background image and text panels are typeset to generate a target image.

[0011] In some embodiments, there are multiple candidate text templates, and selecting a target template from preset candidate text templates according to the text item includes:

[0012] Obtain multiple sample texts contained in each candidate text template;

[0013] The distances between the text item and each sample text are calculated respectively, and a target template is selected from the plurality of candidate text templates according to the distances.

[0014] In some embodiments, the sample text includes text items in multiple dimensions, and calculating the distance between the text item and the sample text includes:

[0015] Obtaining text items of the text item and the sample text in each dimension, and calculating the distance between the text items in each dimension respectively;

[0016] The distance between the text item and the sample text is calculated based on a preset penalty factor and the distance between the text items in each dimension.

[0017] In some embodiments, determining the target position of the text block in the background image according to the pixel information of the saliency map includes:

[0018] Calculating, according to the pixel information of the saliency map, a balanced energy value when the text block is located at each preset candidate position in the background image;

[0019] According to the balance energy value, a target position of the text block in the background image is screened out from the candidate positions.

[0020] In some embodiments, calculating the equilibrium energy value when the text block is located at a preset candidate position in the background image according to the pixel information of the saliency map includes:

[0021] Acquire pixel information when the text block is located at the candidate position;

[0022] Calculating energy parameters according to the pixel information of the saliency map and the text plate;

[0023] The balanced energy value when the text block is located at the candidate position is calculated based on the preset weight and the energy parameter.

[0024] In some embodiments, the energy parameter includes a conflict parameter, and the energy parameter is calculated based on the pixel information of the saliency map and the text block, including:

[0025] According to the pixel information of the saliency map and the text block, counting the overlapping pixels of the saliency map and the text block;

[0026] According to the overlapping pixels, the conflict value between the saliency map and the text plate is calculated to obtain a conflict parameter.

[0027] In some embodiments, the energy parameter includes a blanking parameter, and the energy parameter is calculated based on the pixel information of the saliency map and the text block, including:

[0028] According to the pixel information of the salient map and the text block, counting the blank pixels in the salient map and the text block;

[0029] The blanking rate is calculated according to the blanking pixels to obtain a blanking parameter.

[0030] In some embodiments, the energy parameter includes an alignment parameter, and the energy parameter is calculated based on the pixel information of the saliency map and the text block, including:

[0031] Calculating a deviation value between the saliency map and the text plate in a preset direction according to pixel information of the saliency map and the text plate;

[0032] An alignment parameter is calculated according to a preset coefficient corresponding to the preset direction and the deviation value.

[0033] In some embodiments, the step of calculating the equilibrium energy value when the text block is located at the candidate position according to the preset weight and the energy parameter includes:

[0034] Obtaining the center position information of the background image and the text plate, and calculating the center parameter according to the center position information;

[0035] The balanced energy value when the text block is located at the candidate position is calculated based on the preset weight, the energy parameter and the center parameter.

[0036] In some embodiments, the step of selecting a target template from preset candidate text templates according to the text item may include:

[0037] The subject information is obtained, and candidate text templates are screened out from a preset template library according to the subject information.

[0038] In some embodiments, obtaining the background image includes:

[0039] Acquire keyword information, and select multiple candidate background images from a preset background library according to the keyword information;

[0040] Obtaining description texts corresponding to each candidate background image, and calculating the similarity between each description text and the text item;

[0041] A background image is selected from the plurality of candidate background images according to the similarity.

[0042] In some embodiments, selecting a background image from the plurality of candidate background images according to the similarity includes:

[0043] Obtain the performance value corresponding to each candidate background image;

[0044] Calculate the total score of each candidate background image according to the performance value and similarity;

[0045] A background picture is selected from the plurality of candidate background pictures according to the total score of each candidate background picture.

[0046] The embodiment of the present invention further provides an image generating device, comprising:

[0047] A template unit, used to obtain an input text item and select a target template from preset candidate text templates according to the text item;

[0048] A section unit, used for typeset the text item according to the target template to generate a text section;

[0049] A saliency map unit, used to obtain a background image and extract a saliency map of the background image;

[0050] a position unit, used to obtain pixel information of the saliency map, and determine a target position of the text plate in the background image according to the pixel information of the saliency map;

[0051] The image unit is used to typeset the background image and text panels according to the target position to generate a target image.

[0052] An embodiment of the present invention further provides a storage medium, wherein the storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in the image generating method provided by any embodiment of the present invention.

[0053] The embodiment of the present invention obtains an input text item, and selects a target template from a preset candidate text template according to the text item, so that the obtained target template has a high matching degree with the text item; then, typeset the text item according to the target template to generate a text block; and obtains a background image, extracts a saliency map of the background image; then, obtains pixel information of the saliency map, and determines the target position of the text block in the background image according to the pixel information of the saliency map; and typeset the background image and the text block according to the target position to generate a target image. The scheme selects a suitable text template for each input text item, thereby obtaining a target template that best matches the text item, and realizing personalized copy typesetting. Moreover, the target position of the text block is determined according to the saliency map in the background image, so that the text block and the background image are typeset, so that the text block reaches a balance or reaches an optimal fusion state with the saliency map in the background image, thereby realizing the visual balance of the target image and improving the overall aesthetics of the image typesetting. As a result, the solution realizes the automatic layout of text content and text position, as well as the automatic generation of graphic images, without the need for manual settings or adjustments by users, saving users' time and energy, and improving the overall effect of the image while increasing the degree of automation in graphic image production. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1a is a schematic diagram of a scenario of an information interaction system provided by an embodiment of the present invention;

[0056] Figure 1b is a flowchart of an image generating method provided by an embodiment of the present invention;

[0057] Figure 1c is a schematic diagram of inputting text items on a copywriting page provided by an embodiment of the present invention;

[0058] Figure 1d It is a schematic diagram of a target image provided by an embodiment of the present invention.

[0059] Figure 2a is a flow chart of another image generating method provided by an embodiment of the present invention;

[0060] Figure 2b is a schematic diagram of subject information input provided by an embodiment of the present invention;

[0061] Figure 3aIt is a schematic diagram of a target image production application scenario process provided by an embodiment of the present invention;

[0062] Figure 3b is a schematic diagram of a target template provided by an embodiment of the present invention;

[0063] Figure 3c is a schematic diagram of a text section provided by an embodiment of the present invention;

[0064] Figure 3d is a schematic diagram of a background image provided by an embodiment of the present invention;

[0065] Figure 3e is a schematic diagram of a target image provided by an embodiment of the present invention;

[0066] Figure 4a is a schematic diagram of the structure of an image generating device provided by an embodiment of the present invention;

[0067] Figure 4b is a schematic structural diagram of another image generating device provided by an embodiment of the present invention;

[0068] Figure 5 It is a schematic diagram of the structure of a server provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0070] Embodiments of the present invention provide an image generating method, an image generating device and a storage medium.

[0071] The embodiment of the present invention provides an information interaction system, which includes an image generation device provided by any embodiment of the present invention, and the image generation device can be integrated in a server or other device; in addition, the system can also include other devices, such as a client, etc. The client can be a terminal or a personal computer (PC, Personal Computer) or other devices.

[0072] refer to Figure 1a , an embodiment of the present invention provides an information interaction system, including an image generating device and a client.

[0073] When a user creates a graphic image, such as a print advertisement, the user uses the client to input the text item. The client sends the text item input by the user to the image generation device. The image generation device obtains the text item and selects a target template from the preset candidate text templates according to the text item; then, the text item is typeset according to the target template to generate a text section. In addition, the user can also use the client to upload a background image to the image generation device, or select a background image from the candidate background images provided by the image generation device. The image generation device obtains the background image and extracts a salient image of the background image.

[0074] Then, the image generation device obtains pixel information of the saliency map, determines the target position of the text block in the background image according to the pixel information of the saliency map, and typesets the background image and the text block according to the target position to generate a target image.

[0075] The solution selects a suitable text template for each input text item, thereby obtaining the target template that best matches the text item and realizing personalized text layout. In addition, the target position of the text block is determined based on the salient image in the background image, so that the text block and the background image are laid out so that the text block and the salient image in the background image are balanced or optimally integrated, achieving the visual balance of the target image and improving the overall aesthetics of the image layout. As a result, the solution realizes the automatic layout of the text content and the position of the text, without the need for manual settings or adjustments by the user, saving the user's time and energy, and improving the automation of poster image production while ensuring the overall effect of the poster.

[0076] like Figure 1b As shown, the specific process of the image generation method can be as follows:

[0077] 101. Obtain an input text item, and select a target template from preset candidate text templates based on the text item.

[0078] For example, when a user is creating a target image such as a print advertisement, he or she can input text items on a copy page provided by the image generation device through a mobile phone, tablet computer, or personal computer. The text items input by the user may include text items of one or more dimensions. The dimensions may be the category, level, priority, and / or sorting of the text items, which may be flexibly configured according to actual needs.

[0079] For example Figure 1cThe text items entered by the user on the copy page include text items of multiple dimensions, namely: main copy 1, sub-copy 1, main copy 2 and sub-copy 2. Among them, "Last winter" is a text item of the dimension "main copy 1", "Host: Xiaobei" is a text item of the dimension "sub-copy 1", "I fell in love with a girl" is a text item of the dimension "main copy 2", and the text item of the dimension "sub-copy 2" is empty.

[0080] The image generating device obtains a text item input by a user.

[0081] Then, the image generation device selects a target template that matches the text item from the preset candidate text templates based on the amount or content of text contained in the text item to achieve the optimal text typesetting effect, which will be explained in detail with examples below.

[0082] Among them, the candidate text template can be pre-set, or pre-screened according to the theme of the target image, and can be flexibly configured according to actual needs. There can be one or more candidate text templates. If there is only one candidate text template, the candidate text template is used as the target template. This embodiment is described below with multiple candidate text templates.

[0083] The candidate text template includes the layout information of the text item, such as: the number of arranged rows, paragraph spacing, line spacing, font, font size and / or alignment, etc. Among them, the number of arranged rows refers to the number of rows in which the text items in the text item are arranged in the text block. Paragraph spacing refers to the number of blank characters at the beginning of each line of text, for example, no blank, or 2 or more blank characters, etc. The paragraph spacing of each line of text can be the same or different. Line spacing refers to the distance between each line of text, and the distance between different lines can be the same or different. Font refers to the font of each line of text, such as Kaiti or Songti, etc. The font of each line of text can be the same or different. Font size refers to the font size of each line of text, such as size 4 or size 5, etc. The size of each line of text can be the same or different. Alignment refers to the alignment of each line of text with other text and / or text blocks, including left alignment, center alignment and / or right alignment, etc.

[0084] This achieves a rich variety of text layout methods.

[0085] For example, in some embodiments, the image generation device selects a target template that is compatible with the text item based on the number of texts in the preset candidate text templates. As an implementation, the image generation device may obtain the number of text entries in the text item, and select a template containing the same number of text entries from the preset candidate text templates as the target template. For example, if the number of text entries in the text item is n, the image generation device may extract the number of text entries contained in each candidate text template, and select a candidate text template containing n text entries as the target template, where n is a positive integer.

[0086] In some other embodiments, the step of "filtering out a target template from preset candidate text templates based on a text item" may include: obtaining multiple sample texts contained in each candidate text template; respectively calculating the distance between the text item and each sample text, and selecting a target template from multiple candidate text templates based on the distance.

[0087] Specifically, the image generation device may respectively obtain sample texts contained in each candidate text template, wherein the sample texts may contain sample items of one or more dimensions.

[0088] Then, the image generation device calculates the distance between the text item and each sample text respectively, wherein the distance between the text item and the sample text is used to describe the similarity between the text item and the sample text.

[0089] For example, when the image generating device screens the target template based on the text content contained in the text item, the image generating device may calculate the semantic similarity, Hamming distance, etc. between the text item and the sample text as the distance between the two.

[0090] As another embodiment, when the image generation device selects the target template based on the amount of text contained in the text item, the text amount similarity between the text item and the sample text can be calculated as the distance between the two. Specifically, taking the sample text of any candidate text template as an example, the step of "calculating the distance between the text item and the sample text" can include: obtaining the text items of the text item and the sample text in each dimension, and calculating the distance between the text items in each dimension respectively; calculating the distance between the text item and the sample text according to the preset penalty factor and the distance between the text items in each dimension.

[0091] The image generating device counts the dimensions where the text items in the sample text are located, and the image generating device counts the dimensions where the text items in the text items are located.

[0092] Then, the image generation device merges and deduplicates the text entry dimensions contained in the sample text and text items, and obtains all the dimensions contained in the sample text and text items. For example, the text entry dimensions contained in the sample text are main copy one, sub-copy one, and main copy two, and the text entry dimensions contained in the text item are main copy one and main copy two. Then, the image generation device merges the text entry dimensions contained in the sample text and text items, and obtains 2 main copy ones, 1 sub-copy one, and 2 main copy twos. Then, the image generation device deduplicates the merged text entry dimensions, and obtains 1 main copy one, 1 sub-copy one, and 1 main copy two. Thus, the image generation device obtains all the dimensions contained in the sample text and text items: main copy one, sub-copy one, and main copy two.

[0093] Then, the image generation device obtains the text items and the text entries of the sample text in each dimension according to all the dimensions obtained, and calculates the distance between the text entries in each dimension.

[0094] For example, the image generation device can count the number of characters in the text items and sample texts in each dimension, for example, the number of Chinese characters, the number of English characters, or the number of all characters. If the text item or sample text has no text entry in a certain dimension, the number of characters in the text entry in that dimension is set to zero.

[0095] Then, the image generation device calculates the difference in the number of text item characters between the text item and the sample text in the same dimension, and uses the absolute value of the difference in the number as the text item distance in the dimension. Thus, the image generation device calculates the text item distance between the text item and the sample text in each dimension.

[0096] Then, the image generation device multiplies the text entry distance of each dimension by the corresponding preset penalty factor, respectively, to obtain the reference distance between the text item and the sample text in each dimension. Among them, the penalty factor can be a constraint function or a weight value. In this embodiment, the penalty factor is used as an example of a weight value, and the value of the penalty factor can be pre-configured, and the value of the penalty factor can be proportional to the importance of its corresponding dimension. For example, the importance of the dimension "main copy one" is higher than that of the "main copy two", then the penalty factor value corresponding to the dimension "main copy one" can be greater than the penalty factor value corresponding to the "main copy two". Thus, through the penalty factor, the influence of the text entry distance of each dimension on the distance between the text item and the sample text is highlighted or weakened, thereby realizing personalized customization of the target template screening, which is more in line with customer needs.

[0097] In some embodiments, the image generation device obtains preset penalty factors for each dimension corresponding to the candidate text template based on the candidate text template where the sample text is located. Different candidate text templates may have different penalty factors corresponding to the same dimension. Of course, the image generation device can also detect whether there are missing text entries in each dimension. For example, if a text item or sample text is missing in the text entry of any dimension, the penalty factor corresponding to the dimension is modified to a preset value to control the impact of the missing text in the dimension.

[0098] Then, the image generating device adds up the reference distances between the text item and the sample text in each dimension to obtain the distance between the text item and the sample text.

[0099] For example, the text item is text1 and the sample text is text2. The number of words in the text item for dimension x is x. 1 , the number of words in the text entry of dimension x in the sample text is x 2 ; The number of words in the text item of dimension y is y 1 , the number of words in the text entry of dimension y in the sample text is y 2 ; The number of words in the text item of dimension z is z 1 , the number of words in the text entry of dimension z in the sample text is z 2 ; ...; The number of words in the text item of dimension w is w 1 , the number of words in the text entry of dimension w in the sample text is w 2 .

[0100] The penalty factor of the default dimension x is η x , the penalty factor of dimension y is η y , the penalty factor of dimension z is η z , ..., the penalty factor of dimension w is η w .

[0101] Then the distance D(text1, text2) between text1 and text2 can be calculated using the following formula:

[0102] D(text1, text2) = η x ×|x 1 -x 2 |+η y ×|y 1 -y 2 |+η z ×|z 1 -z 2 |+……+η w ×|w 1 -w 2 |.

[0103] Thus, the image generation device calculates the distance between the text item and each sample text respectively, and the distance represents the similarity between the text item and the number of texts in the sample text. Thus, the distance data corresponding to each sample text is obtained.

[0104] Then, the image generating device determines the correspondence between each candidate text template and the distance data according to the correspondence between each sample text and each candidate text template, and the correspondence between each sample text and the distance data.

[0105] After obtaining the distance, the image generation device selects a target template from multiple candidate text templates according to a preset screening rule. For example, the preset screening rule may be to select the candidate text template with the smallest or largest distance value from the text item as the target template from multiple preset candidate text templates.

[0106] Thus, the image generating device obtains the target template.

[0107] 102. Layout the text items according to the target template to generate text blocks.

[0108] After obtaining the target template, the image generating device typeset the text items according to the typesetting information contained in the target template, generates text blocks for filling in the background image, and generates the target image.

[0109] The typesetting information included in the target template may specifically include: number of rows, paragraph spacing, line spacing, font, font size and / or alignment, etc.

[0110] Take the example that the text item contains three dimensions of text entries: text one of the first dimension, text two of the second dimension, and text three of the third dimension. The sample text of the target template also contains the text entries of the above three dimensions, arranged in three rows, and the font sizes of the first, second, and third rows are one, two, and three respectively.

[0111] The image generation device arranges the text items of the three dimensions in the text item in rows in sequence according to the preset dimension priority, and obtains a text module consisting of three lines of text. For example, according to the priority of the dimensions from high to low, the arrangement order of each dimension is the first dimension, the second dimension and the third dimension, then the image generation device arranges text one, text two and text three in three rows in sequence, with text one in the first row, text two in the second row, and text three in the third row.

[0112] Furthermore, the image generating device sets the font sizes of the text one, the text two and the text three to size one, size two and size three respectively.

[0113] It should be noted that the text section can be a section of fixed size and can be directly displayed on the background image. Of course, the background color of the text section can be pre-configured to be transparent to avoid blocking the content in the background image. Of course, the background color of the text section can also be other preset colors to highlight the text section.

[0114] 103. Obtain a background image, and extract a salient map of the background image.

[0115] It should be noted that step 103 may be executed before or after steps 101 and 102, or may be executed simultaneously. That is, the image generating device may execute step 103 while, before or after executing steps 101 and 102.

[0116] The background image is the background image of the target image. For example, the user can upload the background image, or the user can select the background image from the candidate images provided by the image generation device.

[0117] Saliency map refers to the salient areas in the image, that is, the areas that the viewer is interested in. When seeing an image, the viewer will automatically process the areas of interest and selectively ignore the areas of no interest. These areas of interest to the viewer are called salient areas.

[0118] After the image generation device obtains the background image, it needs to extract the salient map therein.

[0119] For example, we can use deep supervised salient object detection with short connections to extract salient maps in background images. Alternatively, we can use a bottom-up or top-down computational model to perform saliency analysis on background images to capture salient objects that the human eye may notice, and then segment the background image based on the salient objects to obtain a salient map containing the salient objects.

[0120] 104. Obtain pixel information of the saliency map, and determine a target position of the text block in the background image according to the pixel information of the saliency map.

[0121] The pixel information of the saliency map includes pixel information in the saliency map, such as the coordinates of the pixels of the saliency map in the background image and / or the color information of the pixels.

[0122] For example, the image generation device can calculate the balance energy value of the text block when it is located at each preset candidate position in the background image based on the pixel information of the saliency map; and select the target position of the text block in the background image from each candidate position based on the balance energy value. The details are as follows:

[0123] 1. Calculation of equilibrium energy value.

[0124] The candidate position can be described using its coordinates in the background image. For example, a corner of the background image or the center of the background image can be used as the origin of the coordinate system to determine the coordinates of the candidate position in the background image. It should be noted that the candidate position can be any coordinate position in the background image, and this embodiment is illustrated by taking multiple candidate positions as an example.

[0125] The image generation device calculates the balance energy value when the text block is located at each candidate position, wherein the balance energy value quantitatively represents the visual balance and overall aesthetics of the saliency map and the text block and / or the background image.

[0126] As an implementation method, taking any candidate position as an example, the step of "calculating the balance energy value of the text block when it is located at a preset candidate position in the background image based on the pixel information of the saliency map" may include: obtaining the pixel information of the text block when it is located at the candidate position; calculating the energy parameters based on the pixel information of the saliency map and the text block; and calculating the balance energy value of the text block when it is located at the candidate position based on the preset weights and energy parameters.

[0127] The image generation device obtains pixel information when the text block is located at the candidate position, including the coordinates of the pixels in the text block in the background image, and / or the color information of the pixels.

[0128] Then, the image generation device calculates energy parameters based on the pixel information of the saliency map and the text block, wherein the energy parameters include parameters for quantifying the conflict degree, blank degree and / or alignment degree between the saliency map and the text block.

[0129] In this embodiment, the text block is located in area R, and the saliency map is in area I. The energy parameter is calculated as follows:

[0130] 1. Energy parameters include conflict parameters.

[0131] The step of "calculating energy parameters according to pixel information of the salient map and the text block" may include: counting overlapping pixels of the salient map and the text block according to the pixel information of the salient map and the text block; and calculating the conflict value of the salient map and the text block according to the overlapping pixels to obtain the conflict parameter.

[0132] Among them, the conflict parameter quantitatively represents the degree of conflict between the saliency map and the text block display.

[0133] The image generation device can count the pixels where the saliency map and the text block overlap in position according to the coordinates of the pixels in the saliency map and the text block to obtain the overlapping pixels. In order to more clearly define the degree of overlap and remove the light-colored or colorless overlapping parts, the image generation device can only count the overlapping pixels within a fixed color value or a preset color value range when counting the overlapping pixels.

[0134] After obtaining the overlapping pixels, the image generation device can calculate the ratio of the overlapping pixels to the total pixels of the text block as the conflict parameter. Further, the image generation device can calculate the ratio of the overlapping pixels to the number of pixels of the text items in the text block as the conflict parameter.

[0135] For example, the image generating device may use the following formula to calculate the conflict parameter Es(R) when the text block is located in the region R:

[0136]

[0137] Wherein, (x, y) is the pixel coordinate; (x, y)∈R refers to the pixel of the text block located in region R; I(x, y) refers to the pixel located in the saliency map; M(x, y) refers to the pixel with color value M, for example, in this embodiment, M can be 255. ∑I(x, y), (x, y)∈R refers to the number of pixels located in both region R and region I, and ∑M(x, y), (x, y)∈R refers to the number of pixels with color value M in region R.

[0138] The image retrieval device can then calculate the ratio of the overlapping pixels to the number of pixels with a color value of M in the text block as a conflict parameter.

[0139] 2. Energy parameters include blank space parameters.

[0140] The step of "calculating energy parameters according to pixel information of the saliency map and text block" may include: counting blank pixels in the saliency map and text block according to pixel information of the saliency map and text block; calculating blank ratio according to blank pixels to obtain blank parameters.

[0141] The conflict parameter quantitatively characterizes the blank area of ​​the saliency map and the text block. The blank pixel refers to a pixel of a preset color value, which can usually be a color value of white, a light color, and / or a transparent color.

[0142] The image retrieval device can count the pixels of preset color values ​​in the saliency map and the text block according to the color values ​​of the pixels in the saliency map and the text block to obtain the blank pixels.

[0143] Then, the image retrieval device can use the following formula to calculate the blank parameter Eu(R) when the text block is located in the region R:

[0144]

[0145] Wherein, (x, y) is the pixel coordinate; (x, y)∈R refers to the pixel of the text block located in region R; (x, y)∈I refers to the pixel located in the saliency map; N(x, y) is the pixel with color value N, N is a preset value, for example, in this embodiment, N can be 1. ∑N(x, y), (x, y)∈R refers to the number of pixels located in region R and with color value N, ∑N(x, y), (x, y)∈I refers to the number of pixels located in region I and with color value N.

[0146] Thus, the image retrieval device can calculate the ratio of the number of pixels with a color value of N in the text block area and the salient image as a blanking parameter.

[0147] 3. Energy parameters include alignment parameters.

[0148] The step of "calculating energy parameters according to pixel information of the salient map and the text block" may include: calculating the deviation value of the salient map and the text block in a preset direction according to the pixel information of the salient map and the text block; and calculating the alignment parameter according to the preset coefficient and the deviation value corresponding to the preset direction.

[0149] The alignment parameter quantitatively represents the alignment of the salient image and the text block in a preset direction. The preset direction includes horizontal and / or vertical. In this embodiment, the preset direction includes horizontal and vertical for example.

[0150] The image retrieval device can count the number of pixels of the saliency map and the text block aligned horizontally and vertically, calculate the horizontal and vertical deviation rates, and then calculate the alignment parameter according to the preset coefficient and deviation rate. For example, the image retrieval device can use the following formula to calculate the alignment parameter Em(R) when the text block is located in area R:

[0151]

[0152] Where (x, y) is the pixel coordinate; y∈Ry refers to the pixel with different horizontal coordinates but the same vertical coordinates as the pixel in the text block; x∈R x , Refers to the pixel with the same horizontal coordinate as the pixel in the text block, but different vertical coordinate. I(x, y) refers to the pixel in the saliency map, ∑I(x, y), Refers to the number of pixels in the saliency map whose horizontal coordinates are different from those in the text block, but whose vertical coordinates are the same, ∑I(x, y), Refers to the number of pixels in the saliency map with the same horizontal coordinates as the pixels in the text block, but different vertical coordinates. O(x, y) is the pixel with a color value of O, where O is a preset value. For example, in this embodiment, O can be 255, ∑0(x, y), Refers to the number of pixels whose horizontal coordinates are different from those in the saliency map and the text block, but whose vertical coordinates are the same and whose color value is O, ∑0(x, y), Refers to the number of pixels whose horizontal coordinates are the same as those in the text block, but whose vertical coordinates are different and whose color value is 0.

[0153] and are the horizontal and vertical deviation rates between the salient map and the text plate, μ m1 and μ m2 They are the preset coefficients corresponding to the horizontal and vertical directions respectively.

[0154] Thus, the image retrieval device can calculate the alignment parameters.

[0155] After obtaining the energy parameters, the image retrieval device can calculate the equilibrium energy value when the text block is located at the current candidate position according to the preset weights corresponding to the energy parameters and the energy parameters. Among them, the preset weights can be flexibly configured according to actual needs. For example, the value of the weight is proportional to the importance of each energy parameter. It should be noted that the sum of the preset weights corresponding to each energy parameter is equal to 1.

[0156] The energy parameter may include a conflict parameter, a blank parameter and / or an alignment parameter. In this embodiment, an example is given in which the energy parameter includes the conflict parameter, the blank parameter and the alignment parameter.

[0157] The image retrieval device can use the following formula to calculate the equilibrium energy E(R) when the text block is located in the region R:

[0158] E(R)=α×Es(R)+β×Eu(R)+γ×Em(R).

[0159] Among them, α, β and γ are the preset weights corresponding to the conflict parameter, the blank parameter and the alignment parameter respectively, and the sum of the three is equal to 1.

[0160] Thus, the image retrieval device can respectively calculate the balance energy value when the text block is located at each candidate position.

[0161] In some embodiments, the alignment of the text block and the background image may also be considered. Then the step of "calculating the balance energy value of the text block when it is located at the candidate position according to the preset weight and energy parameters" may include: obtaining the center position information of the background image and the text block, and calculating the center parameter information according to the center position; calculating the balance energy value of the text block when it is located at the candidate position according to the preset weight, energy parameter and center parameter.

[0162] The center position of the background image refers to the geometric center of the background image, which can be represented by coordinates. The center position of the text block refers to the geometric center of the text block when it is located at the candidate position, which can be represented by coordinates.

[0163] The image generation device obtains the center coordinates of the background image and the text block respectively as the center position information. The center coordinates of the background image are taken as (C x , C y ), the center coordinate of the text block is (x c ,y c ).

[0164] Then, the image retrieval device can calculate the center parameter Ec(R) using the following formula:

[0165]

[0166] Among them, μ c1 and μ c2 They are the preset coefficients corresponding to the horizontal and vertical directions respectively.

[0167] Taking the energy parameters including conflict parameter, blank parameter, alignment parameter and center parameter as an example, the image retrieval device can use the following formula to calculate the equilibrium energy E(R) when the text block is located in area R:

[0168] E(R)=α×Es(R)+β×Eu(R)+γ×Em(R)+δ×Ec(R).

[0169] Among them, a, β, γ and δ are the preset weights corresponding to the conflict parameter, blank parameter, alignment parameter and center parameter respectively, and the sum of the four is equal to 1.

[0170] Thus, the image retrieval device can respectively calculate the balance energy value when the text block is located at each candidate position.

[0171] 2. Determination of target location.

[0172] After obtaining the balance energy value, as an implementation, the image generation device uses the candidate position with the smallest balance energy value among the candidate positions as the target position. The smallest balance energy value indicates that when the text block is located at this position, the visual balance and overall aesthetics of the salient image and the text block and / or the background image in the target image can be optimized.

[0173] Thus, the image generating device obtains the target position of the text block in the background image.

[0174] 105. According to the target position, the background image and text panels are arranged to generate the target image.

[0175] For example, the image generation device sets the text block at a target position in the background image to complete the typesetting. Then, the image generation device combines the background image and the text block to generate a target image.

[0176] like Figure 1d As shown, in the target image generated by the image generating device, the salient image is a phonograph, and the text items include two text items, namely, music afternoon tea for literary youth and music radio. It can be seen that the text items are carefully typeset, and the typeset of the salient image and the text items achieves visual balance and overall harmonious beauty.

[0177] As can be seen from the above, the embodiment of the present invention obtains the input text item, and selects the target template from the preset candidate text templates according to the text item, so that the obtained target template has a high matching degree with the text item; then, the text item is typeset according to the target template to generate a text block; and, a background image is obtained, and a saliency map of the background image is extracted; then, the pixel information of the saliency map is obtained, and the target position of the text block in the background image is determined according to the pixel information of the saliency map; according to the target position, the background image and the text block are typeset to generate a target image. The scheme selects a suitable text template for each input text item, so as to obtain the target template that best matches the text item, and realizes personalized text typesetting. Moreover, the target position of the text block is determined according to the saliency map in the background image, so as to typeset the text block and the background image, so that the text block reaches a balance or reaches the optimal fusion state with the saliency map in the background image, realizes the visual balance of the target image, and improves the overall aesthetics of the image typesetting. As a result, the solution realizes the automatic layout of text content and text position, as well as the automatic generation of graphic images, without the need for manual settings or adjustments by users, saving users' time and energy, and improving the overall effect of the image while increasing the degree of automation in graphic image production.

[0178] The method described in the above embodiments is further described in detail below with examples.

[0179] For example, refer to Figure 2aIn this embodiment, the image generating device will be specifically integrated into a server for illustration.

[0180] 201. Obtain subject information, and select candidate text templates from a preset template library according to the subject information.

[0181] For example, when creating a target image such as a poster, the user uses a mobile phone or other device to input the theme of the target image. Alternatively, the user can select the theme of the target image from the candidate themes provided by the server. The theme of the target image can be a style theme and / or a scene theme, such as small fresh, billboard, etc.

[0182] The server obtains the subject information. It should be noted that the subject information may include one or more subject items. Figure 2b As shown, the server provides two levels of topics, namely, primary classification and secondary classification. Each level has corresponding candidate topics. The user selects the emotion topic type in the primary classification, and the topics in the secondary classification are empty.

[0183] Then, the server selects the text template corresponding to the subject information from the preset template library as the candidate text template. The preset template library contains the text templates corresponding to the preset subjects and records the mapping relationship between the subjects and the text templates.

[0184] For example, the subject information includes a subject item, and the server selects a text template corresponding to the subject item as a candidate text template according to a subject-template mapping relationship preset in the template library.

[0185] For example, if the subject information includes multiple subject items, the server selects the text template corresponding to each subject item according to the subject-template mapping relationship preset in the template library. Then, among the text templates corresponding to each subject item, the server selects the text templates that are repeated twice or more as candidate text templates.

[0186] Therefore, it is possible to determine candidate templates for the copy according to the theme of the target image, which is more in line with the actual demand of the target image theme for the copy.

[0187] 202. Obtain an input text item, and select a target template from candidate text templates based on the text item.

[0188] The specific implementation method can refer to the description of step 101 in the above-mentioned image generation method embodiment, which will not be repeated here.

[0189] 203. Typeset the text items according to the target template to generate text blocks.

[0190] The specific implementation method can refer to the description of step 102 in the above-mentioned image generation method embodiment, which will not be repeated here.

[0191] 204. Obtain keyword information, and select multiple candidate background images from a preset background library according to the keyword information.

[0192] It should be noted that steps 204 , 205 , 206 and 207 may be executed before, after or simultaneously with steps 201 - 203 .

[0193] For example, a user can use a mobile phone or other device to enter keywords to filter background images. It should be noted that the user can enter one or more keywords, and the keywords can be the same as the subject. Alternatively, the image generation device can also perform word segmentation on the input text item to extract keyword information. For example, the image generation device performs word segmentation on the text item according to a preset dictionary and obtains the part of speech of the word segmentation; then, the image generation device configures the word segmentation with a preset part of speech category as a keyword. The preset category can be nouns, adjectives and / or verbs, such as nouns such as cake, adjectives such as fresh, verbs such as dance, etc.

[0194] The server obtains the keyword information input by the user, and selects a plurality of candidate background images from a preset background library according to the keyword information. The preset background library contains background images corresponding to preset keywords, and records the mapping relationship between keywords and background images.

[0195] For example, if the keyword information includes a keyword, the server selects a background image corresponding to the keyword as a candidate background image according to a keyword-background image mapping relationship preset in the background library.

[0196] For example, if the keyword information includes multiple keywords, the server selects the background image corresponding to each keyword according to the keyword-background image mapping relationship preset in the background library. Then, among the background images corresponding to each keyword, the background images repeated twice or more are selected as candidate background images.

[0197] Therefore, it is possible to determine candidate background images according to keywords, thereby improving the flexibility and diversity of background image selection.

[0198] 205. Obtain the description text corresponding to each candidate background image, and calculate the similarity between each description text and the text item.

[0199] The description text corresponding to the candidate background image may be preset, or may be configured by the server according to the objects and / or colors contained in the background image after the server identifies the candidate background image, and may be flexibly configured according to actual needs.

[0200] Taking the description text corresponding to any candidate background image as an example, the server performs word segmentation on the text item and the description text of the candidate background image to obtain the word vector space. Then, based on the word vector space of the text item and the description text, the similarity between the two is calculated. For example, the cosine algorithm can be used to calculate the similarity, or the likelihood calculation method can be used to calculate the similarity. In this way, the server can calculate the similarity between the text item and the description text of each candidate background image.

[0201] 206. Select a background image from multiple candidate background images based on similarity.

[0202] For example, the server may select the description text with the largest similarity value as the most matching text, and determine the candidate background image corresponding to the description text as the background image of the target image.

[0203] In some embodiments, the historical performance of the candidate background images can also be taken into consideration, and the step of "selecting a background image from multiple candidate background images based on similarity" can include: obtaining the performance value corresponding to each candidate background image; calculating the total score of each candidate background image based on the performance value and the similarity; and selecting a background image from multiple candidate background images based on the total score of each candidate background image.

[0204] The performance value quantitatively represents the performance capability of the candidate background image, for example, it may be a historical click rate or a user's like rate, etc. As an implementation method, the server may obtain the historical click count of the candidate background image, and perform regularization on the historical click count to obtain the historical click rate of the candidate background image.

[0205] Taking any candidate background image as an example, the server obtains the similarity between the description text of the candidate background image and the text item, as well as the performance value of the candidate background image. Then, the server calculates the product of the similarity and the performance value, and uses the obtained product as the total score of the candidate background image. In this way, the server can calculate the total score of each candidate background image respectively.

[0206] Then, the server may select the candidate background image with the largest total score value from the multiple candidate background images as the background image of the target image.

[0207] Therefore, the background image selected by the server comprehensively considers the degree of fit with the target theme and the performance ability of the background image, thereby increasing the attractiveness of the target image to the audience and achieving good commercial results.

[0208] 207. Extract saliency map of background image.

[0209] The specific implementation method can refer to the description of step 103 in the above-mentioned image generation method embodiment, which will not be repeated here.

[0210] 208. Obtain pixel information of the saliency map, and determine the target position of the text block in the background image according to the pixel information of the saliency map.

[0211] The specific implementation method can refer to the description of step 104 in the above-mentioned image generation method embodiment, which will not be repeated here.

[0212] 209. According to the target position, the background image and text panels are arranged to generate the target image.

[0213] The specific implementation method can refer to the description of step 105 in the above-mentioned image generation method embodiment, which will not be repeated here.

[0214] As can be seen from the above, in the embodiment of the present invention, the user can select text templates and background images according to the theme and keywords, respectively, so as to achieve the diversity and flexibility of making the target image. Moreover, the selected text templates and background images are more in line with the user's preferences and adapt to the application scenario. The text templates and background images are also more in line with the theme to be conveyed by the target image, highlighting the text information to be conveyed, and improving the commercial value and practical value of the target image.

[0215] The method described in the above embodiments is further described in detail below with examples.

[0216] For example, refer to Figure 3a In this embodiment, the image generating device will be specifically integrated into a server for illustration.

[0217] (I) Configure the template library.

[0218] Users can configure text templates for copywriting and store them in the template library.

[0219] In some embodiments, the user may also configure a mapping relationship between a text template and a theme, and store the mapping relationship in a template library.

[0220] Alternatively, the user can upload a graphic image such as a poster to the server. The server extracts the text plate in the graphic image, and then configures a text template according to the format information and / or color information of the sample text in the text plate, and stores the configured text template in the template library.

[0221] In some embodiments, when uploading a text image, the user can also set the theme corresponding to the text image. Thus, the server configures the mapping relationship between the text template and the theme in the template library according to the theme corresponding to the text image.

[0222] (ii) Obtain text items and topics.

[0223] When making a target image, the user can input the text to be displayed in the target image, that is, the text item, to the server. In addition, the user can input the theme information so as to match a more suitable text template.

[0224] The server receives the text item and subject information entered by the user.

[0225] (III) Determine the target template.

[0226] The server selects multiple candidate text templates from the template library based on the topic information and the mapping relationship between the text templates and the topics in the template library. Thus, the server selects the target template with the highest matching degree with the text item from the multiple candidate text templates based on the distance between the text item and the sample text in the candidate text template.

[0227] The target template contains information such as the number of rows, line spacing, font size, and alignment of the text items. Figure 3b The target template shown includes a first line of text and a second line of text, etc. The first line of text and the second line of text have different font sizes and different alignments.

[0228] (iv) Generate text sections.

[0229] After obtaining the target template, the server typeset the text items according to the target template to generate text blocks.

[0230] based on Figure 3b The target template shown is Figure 3c The text section shown includes a first text entry and a second text entry in the text item.

[0231] In some embodiments, the background of the text panel can be set to a transparent color.

[0232] (V) Extract the salient map of the background image.

[0233] The server may obtain the background image uploaded or selected by the user, and then extract the saliency map of the background image.

[0234] like Figure 3d The background image shown in the figure has its salient map identified.

[0235] (6) Determine the target position based on the balance energy value.

[0236] The server may calculate the balance energy value when the text block is located at each preset candidate position in the background image according to the pixel information of the saliency map; and then, according to the balance energy value, select the target position of the text block in the background image from each candidate position.

[0237] In some embodiments, before calculating the balance energy value, the server may select a candidate position of the text panel in the background image according to a preset rule.

[0238] The preset rule may be to select any position in the background image as a candidate position.

[0239] In order to reduce the amount of calculation of the image generation device, for example, the preset rule can be to remove positions that exceed the range of the background image. The server removes the positions that make the text block exceed the range of the background image area according to the size of the text block, and the remaining positions in the background image are available candidate positions.

[0240] For example, the preset rule may be to exclude positions that completely overlap with the salient map. Then, the server excludes positions that make the text block completely or partially within the salient map area according to the size of the text block, and the remaining positions in the background image are available candidate positions.

[0241] In some embodiments, when calculating the balance energy value, taking any candidate position as an example, the server can obtain the pixel information when the text block is located at the candidate position; calculate the energy parameters based on the saliency map and the pixel information of the text block; and calculate the balance energy value when the text block is located at the candidate position based on the preset weights and energy parameters.

[0242] The energy parameter may include a conflict parameter, a blank parameter and / or an alignment parameter.

[0243] (VII) Generate a target image.

[0244] After obtaining the background image, the text block and the target position thereof, the server typesets the background image and the text block according to the target position to generate a target image containing the image and the text.

[0245] based on Figure 3c The text blocks shown and Figure 3d The background image shown in Figure 3e The target image shown contains a saliency map and a text item. The current position of the text item minimizes the balanced energy value, achieves the overall balance and visual beauty of the target image, and also highlights the promotional copy.

[0246] From the above, it can be seen that the embodiment of the present invention selects the text template that is closest to the subject and the content of the text item, thereby realizing the adaptation of the copy template; and, the energy value is used to measure the visual balance and overall aesthetics of the image layout after the fusion of text and graphics, and the optimal position of the copy in the background image is found, so that a good graphic effect can be achieved without manual adjustment by the user, thereby improving the degree of automation of image production.

[0247] In order to better implement the above method, an embodiment of the present invention further provides an image generating device, which can be integrated into a terminal device such as a server, a mobile terminal or a personal computer.

[0248] For example, Figure 4a As shown, the image generation device may include a template unit 401, a plate unit 402, a saliency map unit 403, a position unit 404 and an image unit 405, as follows:

[0249] (a) template unit 401;

[0250] The template unit 401 is used to obtain an input text item and select a target template from preset candidate text templates according to the text item.

[0251] For example, when creating a target image such as a print advertisement, a user can input text items on the copy page provided by the template unit 401 through a mobile phone, a tablet computer, or a personal computer. The text items input by the user can include text items of one or more dimensions. The dimensions can be the category, level, priority, and / or sorting of the text items, which can be flexibly configured according to actual needs.

[0252] The template unit 401 obtains the text item input by the user. Then, the template unit 401 selects a target template that is adapted to the text item from the preset candidate text templates to achieve the optimal text layout effect. Among them, the candidate text templates can be pre-set, or pre-screened according to the theme of the target image, and can be flexibly configured according to actual needs. There can be one or more candidate text templates. If there is only one candidate text template, the candidate text template is used as the target template. This embodiment is illustrated by an example with multiple candidate text templates.

[0253] In some embodiments, the template unit 401 can be specifically used to: obtain multiple sample texts included in each candidate text template; calculate the distance between the text item and each sample text respectively, and select a target template from the multiple candidate text templates according to the distance.

[0254] Specifically, the template unit 401 may respectively obtain the sample texts contained in each candidate text template, wherein the sample texts may contain sample items of one or more dimensions.

[0255] Then, the template unit 401 calculates the distance between the text item and each sample text respectively. The distance between the text item and the sample text is used to describe the similarity between the text item and the sample text. For example, the template unit 401 can calculate the semantic similarity, Hamming distance, etc. between the text item and the sample text as the distance between the two.

[0256] As an implementation method, when the template unit 401 selects the target template based on the amount of text contained in the text item, the similarity between the amount of text in the text item and the sample text can be calculated as the distance between the two. Specifically, taking a sample of any candidate text template text as an example, the template unit 401 can be used to: obtain the text items of the text item and the sample text in each dimension, and calculate the distance between the text items in each dimension respectively; calculate the distance between the text item and the sample text according to the preset penalty factor and the distance between the text items in each dimension.

[0257] The template unit 401 counts the dimensions of the text items in the sample text, and the template unit 401 counts the dimensions of the text items in the text item.

[0258] Then, the template unit 401 merges and removes duplicates of the text item dimensions contained in the sample text and the text item to obtain all the dimensions contained in the sample text and the text item.

[0259] Then, the template unit 401 obtains the text item and the text entry of the sample text in each dimension according to all the dimensions obtained, and calculates the distance between the text entries in each dimension.

[0260] For example, the template unit 401 can count the number of characters in the text entries of each dimension of the text item and the sample text, for example, the number of Chinese characters, the number of English characters, or the number of all characters. If the text item or sample text has no text entry in a certain dimension, the number of characters in the text entry in that dimension is set to zero.

[0261] Then, the template unit 401 calculates the difference in the number of characters between the text item and the sample text in the same dimension, and uses the absolute value of the difference as the text item distance in the dimension. Thus, the template unit 401 calculates the text item distance between the text item and the sample text in each dimension.

[0262] Then, the template unit 401 multiplies the text entry distance of each dimension by the corresponding preset penalty factor to obtain the reference distance between the text item and the sample text in each dimension. The penalty factor can be pre-configured, and the value of the penalty factor can be proportional to the importance of the corresponding dimension. Thus, the penalty factor is used to highlight or weaken the influence of the distance of the text entry in each dimension on the distance between the text item and the sample text, thereby realizing personalized customization of the target template screening to better meet customer needs.

[0263] Then, the template unit 401 adds up the reference distances between the text item and the sample text in each dimension to obtain the distance between the text item and the sample text.

[0264] For example, the text item is text1 and the sample text is text2. The number of words in the text item for dimension x is x. 1 , the number of words in the text entry of dimension x in the sample text is x 2 ; The number of words in the text item of dimension y is y 1 , the number of words in the text entry of dimension y in the sample text is y 2 ; The number of words in the text item of dimension z is z 1 , the number of words in the text entry of dimension z in the sample text is z 2 ; ...; The number of words in the text item of dimension w is w 1 , the number of words in the text entry of dimension w in the sample text is w 2 .

[0265] The penalty factor of the default dimension x is η x , the penalty factor of dimension y is η y , the penalty factor of dimension z is η z , ..., the penalty factor of dimension w is η w .

[0266] Then the distance D(text1, text2) between text1 and text2 can be calculated using the following formula:

[0267] D(text1, text2) = η x ×|x 1 -x 2 |+η y ×|y 1 -y 2 |+η z ×|z 1 -z 2 |+……+η w ×|w 1 -w 2 |.

[0268] Thus, the template unit 401 calculates the distance between the text item and each sample text respectively, and the distance represents the similarity degree between the specific components of the text item and the sample text. Thus, the distance data corresponding to each sample text is obtained.

[0269] Then, the template unit 401 determines the correspondence between each candidate text template and the distance data according to the correspondence between each sample text and each candidate text template, and the correspondence between each sample text and the distance data.

[0270] After obtaining the distance, the template unit 401 selects a target template from multiple candidate text templates according to a preset screening rule. For example, the preset screening rule may be to select a candidate text template with the smallest or largest distance value from the text item as the target template from multiple preset candidate text templates.

[0271] Thus, the template unit 401 obtains the target template.

[0272] (ii) plate unit 402;

[0273] The section unit 402 is used to typeset the text items according to the target template to generate text sections.

[0274] After obtaining the target template, the plate unit 402 typesets the text items according to the typesetting information contained in the target template to generate text plates for filling in the background image to generate the target image.

[0275] The typesetting information included in the target template may specifically include: number of rows, paragraph spacing, line spacing, font, font size and / or alignment, etc.

[0276] (iii) saliency map unit 403;

[0277] The salient map unit 403 is used to obtain a background image and extract a salient map of the background image.

[0278] It should be noted that the execution sequence of the saliency map unit 403 can be before or after the template unit 401 and the plate unit 402. Of course, they can also be executed simultaneously.

[0279] The background image is the background image of the target image. For example, the user can upload the background image, or the user can select the background image from the candidate images provided by the image generation device.

[0280] Saliency map refers to the salient areas in the image, that is, the areas that the viewer is interested in. When seeing an image, the viewer will automatically process the areas of interest and selectively ignore the areas of no interest. These areas of interest to the viewer are called salient areas.

[0281] After the saliency map unit 403 obtains the background image, it needs to extract the saliency map therein.

[0282] For example, the salient map unit 403 may use deep supervised salient object detection with short connections to extract the salient map in the background image. Alternatively, the salient analysis of the background image may be performed through a bottom-up calculation model or a top-down calculation model to capture the salient objects that the human eye may notice, and then the background image may be segmented according to the salient objects to obtain a salient map containing the salient objects.

[0283] (IV) Location unit 404:

[0284] The position unit 404 is used to obtain pixel information of the saliency map and determine the target position of the text block in the background image according to the pixel information of the saliency map.

[0285] The pixel information of the saliency map includes pixel information in the saliency map, such as the coordinates of the pixels of the saliency map in the background image and / or the color information of the pixels.

[0286] For example, the location unit 404 can specifically calculate the balance energy value when the text block is located at each preset candidate position in the background image based on the pixel information of the saliency map; and select the target position of the text block in the background image from each candidate position based on the balance energy value. Specifically, as follows:

[0287] 1. Calculation of equilibrium energy value.

[0288] The candidate position can be described using its coordinates in the background image. For example, a corner of the background image or the center of the background image can be used as the origin of the coordinate system to determine the coordinates of the candidate position in the background image. It should be noted that the candidate position can be any coordinate position in the background image, and this embodiment is illustrated by taking multiple candidate positions as an example.

[0289] The location unit 404 calculates the balance energy value when the text block is located at each candidate location, wherein the balance energy value quantitatively represents the visual balance and overall aesthetics of the saliency map and the text block and / or the background image.

[0290] As an implementation mode, taking any candidate position as an example, the position unit 404 can be specifically used to: obtain pixel information when the text block is located at the candidate position; calculate energy parameters based on the saliency map and the pixel information of the text block; calculate the balanced energy value when the text block is located at the candidate position based on preset weights and energy parameters.

[0291] The location unit 404 obtains pixel information when the text block is located at the candidate location, including coordinates of pixels in the text block in the background image, and / or color information of the pixels.

[0292] Then, the location unit 404 calculates energy parameters according to the pixel information of the salient map and the text block, wherein the energy parameters include parameters for quantifying the conflict degree, blank degree and / or alignment degree between the salient map and the text block.

[0293] In this embodiment, the text block is located in area R, and the saliency map is in area I. The energy parameter is calculated as follows:

[0294] 1. Energy parameters include conflict parameters.

[0295] The location unit 404 can be specifically used to: count the overlapping pixels of the salient map and the text block according to the pixel information of the salient map and the text block; calculate the conflict value of the salient map and the text block according to the overlapping pixels to obtain the conflict parameter.

[0296] Among them, the conflict parameter quantitatively represents the degree of conflict between the saliency map and the text block display.

[0297] The location unit 404 can count the pixels where the saliency map and the text plate overlap in position according to the coordinates of the pixels in the saliency map and the text plate to obtain overlapping pixels. In order to more clearly define the degree of overlap and remove the light-colored or colorless overlapping parts, the location unit 404 can only count the overlapping pixels within a fixed color value or a preset color value range when counting overlapping pixels.

[0298] After obtaining the overlapping pixels, the location unit 404 can calculate the ratio of the overlapping pixels to the total pixels of the text block as the conflict parameter. Further, the location unit 404 can calculate the ratio of the overlapping pixels to the number of pixels of the text items in the text block as the conflict parameter.

[0299] For example, the location unit 404 may use the following formula to calculate the conflict parameter Es(R) when the text block is located in the region R:

[0300]

[0301] Wherein, (x, y) is the pixel coordinate; (x, y)∈R refers to the pixel of the text block located in region R; I(x, y) refers to the pixel located in the saliency map; M(x, y) refers to the pixel with color value M, for example, in this embodiment, M can be 255. ∑I(x, y), (x, y)∈R refers to the number of pixels located in both region R and region I, and ∑M(x, y), (x, y)∈R refers to the number of pixels with color value M in region R.

[0302] The location unit 404 may calculate the ratio of the overlapping pixels to the pixels with the color value M in the text block as the conflict parameter.

[0303] 2. Energy parameters include blank space parameters.

[0304] The location unit 404 can be specifically used to: count the blank pixels in the salient map and the text block according to the pixel information of the salient map and the text block; calculate the blank ratio according to the blank pixels to obtain the blank parameter.

[0305] The conflict parameter quantitatively characterizes the blank area of ​​the saliency map and the text block. The blank pixel refers to a pixel of a preset color value, which can usually be a color value of white, a light color, and / or a transparent color.

[0306] The location unit 404 may count pixels of preset color values ​​in the saliency map and the text block according to the color values ​​of the pixels in the saliency map and the text block to obtain blank pixels.

[0307] Then, the location unit 404 may use the following formula to calculate the blank parameter Eu(R) when the text block is located in the region R:

[0308]

[0309] Wherein, (x, y) is the pixel coordinate; (x, y)∈R refers to the pixel of the text block located in region R; (x, y)∈I refers to the pixel located in the saliency map; N(x, y) is the pixel with color value N, N is a preset value, for example, in this embodiment, N can be 1. ∑N(x, y), (x, y)∈R refers to the number of pixels located in region R and with color value N, ∑N(x, y), (x, y)∈I refers to the number of pixels located in region I and with color value N.

[0310] Therefore, the location unit 404 can calculate the ratio of the number of pixels with a color value of N in the text block area and the saliency map to obtain the blanking parameter.

[0311] 3. Energy parameters include alignment parameters.

[0312] The position unit 404 can be specifically used to: calculate the deviation value of the salient map and the text block in a preset direction according to the pixel information of the salient map and the text block; and calculate the alignment parameter according to the preset coefficient and the deviation value corresponding to the preset direction.

[0313] The alignment parameter quantitatively represents the alignment of the salient image and the text block in a preset direction. The preset direction includes horizontal and / or vertical. In this embodiment, the preset direction includes horizontal and vertical for example.

[0314] The location unit 404 can count the number of pixels of the saliency map and the text block aligned horizontally and vertically, calculate the horizontal and vertical deviation rates, and then calculate the alignment parameter according to the preset coefficient and deviation rate. For example, the location unit 404 can use the following formula to calculate the alignment parameter Em(R) when the text block is located in the region R:

[0315]

[0316] Where (x, y) is the pixel coordinate; y∈R y Refers to pixels with different horizontal coordinates but the same vertical coordinates as the pixels in the text block; x∈R x , Refers to the pixel with the same horizontal coordinate as the pixel in the text block, but different vertical coordinate. I(x, y) refers to the pixel in the saliency map, ∑I(x, y), Refers to the number of pixels in the saliency map whose horizontal coordinates are different from those in the text block, but whose vertical coordinates are the same, ∑I(x, y), Refers to the number of pixels in the saliency map with the same horizontal coordinates as the pixels in the text block, but different vertical coordinates. O(x, y) is the pixel with a color value of O, where O is a preset value. For example, in this embodiment, O can be 255, ∑0(x, y), Refers to the number of pixels whose horizontal coordinates are different from those in the saliency map and the text block, but whose vertical coordinates are the same and whose color value is O, ∑0(x, y), It refers to the number of pixels whose horizontal coordinates are the same as those in the text block, but whose vertical coordinates are different and whose color value is 0.

[0317] and are the horizontal and vertical deviation rates between the salient map and the text plate, μ mm1 and μ m2 They are the preset coefficients corresponding to the horizontal and vertical directions respectively.

[0318] Thus, the position unit 404 can calculate the alignment parameters.

[0319] After obtaining the energy parameters, the location unit 404 can calculate the equilibrium energy value when the text block is located at the current candidate position according to the preset weights corresponding to the energy parameters and the energy parameters. Among them, the preset weights can be flexibly configured according to actual needs. For example, the value of the weight is proportional to the importance of each energy parameter. It should be noted that the sum of the preset weights corresponding to each energy parameter is equal to 1.

[0320] The energy parameter may include a conflict parameter, a blank parameter and / or an alignment parameter. In this embodiment, an example is given in which the energy parameter includes the conflict parameter, the blank parameter and the alignment parameter.

[0321] The location unit 404 can use the following formula to calculate the equilibrium energy E(R) when the text plate is located in the region R:

[0322] E(R)=α×Es(R)+β×Eμ(R)+γ×Em(R).

[0323] Among them, α, β and γ are the preset weights corresponding to the conflict parameter, the blank parameter and the alignment parameter respectively, and the sum of the three is equal to 1.

[0324] Therefore, the location unit 404 can respectively calculate the balance energy value when the text block is located at each candidate position.

[0325] In some embodiments, the alignment of the text block and the background image can also be considered. The position unit 404 can be specifically used to: obtain the center position information of the background image and the text block, calculate the center parameter information according to the center position; and calculate the balance energy value when the text block is located at the candidate position according to the preset weight, energy parameter and center parameter.

[0326] The center position of the background image refers to the geometric center of the background image, which can be represented by coordinates. The center position of the text block refers to the geometric center of the text block when it is located at the candidate position, which can be represented by coordinates.

[0327] The location unit 404 obtains the center coordinates of the background image and the text block respectively as the center location information. The center coordinates of the background image are taken as (C x , C y ), the center coordinate of the text block is (x c ,y c ).

[0328] Then, the location unit 404 can calculate the center parameter E using the following formula: c (R):

[0329]

[0330] Among them, μ c1 and μ c2 They are the preset coefficients corresponding to the horizontal and vertical directions respectively.

[0331] Taking the energy parameters including the conflict parameter, the blank parameter, the alignment parameter and the center parameter as an example, the location unit 404 can use the following formula to calculate the equilibrium energy E(R) when the text block is located in the region R:

[0332] E(R)=α×Es(R)+β×E μ (R)+γ×Em(R)+δ×Ec(R).

[0333] Among them, a, β, γ and δ are the preset weights corresponding to the conflict parameter, blank parameter, alignment parameter and center parameter respectively, and the sum of the four is equal to 1.

[0334] Therefore, the location unit 404 can respectively calculate the balance energy value when the text block is located at each candidate position.

[0335] 2. Determination of target location.

[0336] After obtaining the balance energy value, as an implementation, the position unit 404 selects the candidate position with the smallest balance energy value among the candidate positions as the target position. The smallest balance energy value indicates that when the text block is located at this position, the visual balance and overall aesthetics of the salient image and the text block and / or the background image in the target image can be optimized.

[0337] Thus, the position unit 404 obtains the target position of the text block in the background image.

[0338] In some embodiments, before calculating the balance energy value, the location unit 404 may select a candidate location of the text plate in the background image according to a preset rule.

[0339] The preset rule may be to select any position in the background image as a candidate position. To reduce the amount of calculation of the image generation device, for example, the preset rule may be to exclude positions beyond the range of the background image, or to exclude positions that completely overlap with the saliency map.

[0340] (iv) image unit 405;

[0341] The image unit 405 is used to typeset the background image and text panels according to the target position to generate a target image.

[0342] For example, the image unit 405 sets the text plate at the target position in the background image to complete the typesetting. Then, the image unit 405 combines the background image and the text plate to generate a target image.

[0343] As can be seen from the above, the embodiment of the present invention obtains the input text item through the template unit 401, and selects the target template from the preset candidate text templates according to the text item, and the obtained target template has a high matching degree with the text item; then, the block unit 402 typesets the text item according to the target template to generate a text block; and the saliency map unit 403 obtains the background image and extracts the saliency map of the background image; then, the position unit 404 obtains the pixel information of the saliency map, and determines the target position of the text block in the background image according to the pixel information of the saliency map; the image unit 405 typesets the background image and the text block according to the target position to generate a target image. The scheme selects a suitable text template for each input text item, thereby obtaining the target template that best matches the text item, and realizing personalized copywriting typesetting. In addition, the target position of the text block is determined according to the saliency map in the background image, so that the text block and the background image are typeset, so that the text block reaches a balance or reaches the optimal fusion state with the saliency map in the background image, realizing the visual balance of the target image, and improving the overall aesthetics of the image typesetting. As a result, the solution realizes the automatic layout of text content and text position, as well as the automatic generation of graphic images, without the need for manual settings or adjustments by users, saving users' time and energy, and improving the overall effect of the image while increasing the degree of automation in graphic image production.

[0344] In addition, if Figure 4b As shown, the image generation device may include a template unit 401, a plate unit 402, a saliency map unit 403, a position unit 404, an image unit 405, and a theme unit 406. The specific implementations of the template unit 401, the plate unit 402, the saliency map unit 403, the position unit 404, and the image unit 405 may refer to the description in the above-mentioned image generation device embodiment, and the theme unit 406 is as follows:

[0345] (vi) Theme Unit 406.

[0346] The theme unit 406 is used to obtain theme information and select candidate text templates from a preset template library according to the theme information.

[0347] For example, when creating a target image such as a poster, the user uses a mobile phone or other device to input the theme of the target image. Alternatively, the user can select the theme of the target image from the candidate themes provided by the theme unit 406. The theme of the target image can be a style theme and / or a scene theme, such as small fresh, billboard, etc.

[0348] The theme unit 406 obtains theme information. It should be noted that the theme information may include one or more theme items. Then, the theme unit 406 selects the text template corresponding to the theme information from the preset template library as a candidate text template. Among them, the preset template library contains the text templates corresponding to the preset themes, and records the mapping relationship between the themes and the text templates.

[0349] Therefore, it is possible to determine candidate templates for the copy according to the theme of the target image, which is more in line with the actual demand of the target image theme for the copy.

[0350] In addition, in some embodiments, the saliency map unit 403 can be specifically used to: obtain keyword information, and filter out multiple candidate background images in a preset background library based on the keyword information; obtain the description text corresponding to each candidate background image, and calculate the similarity between each description text and the text item; and select a background image from multiple candidate background images based on the similarity.

[0351] For example, a user can use a mobile phone or other device to input keywords to filter background images. It should be noted that the user can input one or more keywords, and the keywords can be the same as the theme. Alternatively, the image generation device can also perform word segmentation processing on the input text item to extract keyword information.

[0352] The salient image unit 403 obtains the keyword information input by the user, and selects a plurality of candidate background images from a preset background library according to the keyword information. The preset background library contains background images corresponding to preset keywords, and records the mapping relationship between keywords and background images. Thus, the candidate background images are determined according to the keywords, and the flexibility and diversity of background image selection are improved.

[0353] The description text corresponding to the candidate background image may be preset, or may be configured according to the objects and / or colors contained in the background image after the saliency map unit 403 identifies the candidate background image, and may be flexibly configured according to actual needs.

[0354] Taking the description text corresponding to any candidate background image as an example, the salient map unit 403 performs word segmentation on the text item and the description text of the candidate background image respectively to obtain a word vector space. Then, according to the word vector space of the text item and the description text, the similarity between the two is calculated. For example, the cosine algorithm can be used to calculate the similarity. Thus, the salient map unit 403 can respectively calculate the similarity between the text item and the description text of each candidate background image.

[0355] The saliency map unit 403 may select the description text with the largest similarity value as the most matching text, and determine the candidate background image corresponding to the description text as the background image of the target image.

[0356] In some embodiments, the historical performance of the candidate background images can also be taken into consideration, and the saliency map unit 403 can be specifically used to: obtain the performance value corresponding to each candidate background image; calculate the total score of each candidate background image according to the performance value and similarity; and select a background image from multiple candidate background images according to the total score of each candidate background image.

[0357] The performance value quantitatively represents the performance capability of the candidate background image, for example, it may be a historical click rate or a user's like rate, etc. As an implementation, the saliency map unit 403 may obtain the historical click count of the candidate background image, and perform regularization processing on the historical click count to obtain the historical click rate of the candidate background image.

[0358] Taking any candidate background image as an example, the salient map unit 403 obtains the similarity between the candidate background image description text and the text item, as well as the performance value of the candidate background image. Then, the salient map unit 403 calculates the product of the similarity and the performance value, and uses the obtained product as the total score of the candidate background image. Thus, the salient map unit 403 can calculate the total score of each candidate background image respectively.

[0359] Then, the saliency map unit 403 may select the candidate background map with the largest total score value from the multiple candidate background maps as the background image of the target image.

[0360] Therefore, the background image selected by the saliency map unit 403 comprehensively considers the degree of fit with the target theme and the performance ability of the background image, thereby increasing the attractiveness of the target image to the audience and achieving good commercial results.

[0361] The embodiment of the present invention further provides a server, such as Figure 5 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present invention, specifically:

[0362] The server may include one or more processing core processors 501, one or more computer-readable storage media memories 502, a power supply 503, an input unit 504 and other components. Those skilled in the art will appreciate that Figure 5 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0363] The processor 501 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, the processor 501 performs various functions of the server and processes data, thereby monitoring the server as a whole. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0364] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image generation program, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0365] The server also includes a power supply 503 for supplying power to each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0366] The server may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0367] Although not shown, the server may also include a display unit, etc., which will not be described in detail here. Specifically in this embodiment, the processor 501 in the server will load the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 501 will run the application stored in the memory 502, such as the image generation program, to achieve various functions, as follows:

[0368] The method comprises the following steps: obtaining an input text item, filtering out a target template from preset candidate text templates according to the text item; typeset the text item according to the target template to generate a text block; obtaining a background image, extracting a saliency map of the background image; obtaining pixel information of the saliency map, and determining a target position of the text block in the background image according to the pixel information of the saliency map; typeset the background image and the text block according to the target position to generate a target image.

[0369] In some embodiments, there are multiple candidate text templates, and the processor 501 runs the application stored in the memory 502, and can also implement the following functions:

[0370] A plurality of sample texts included in each candidate text template is obtained; a distance between the text item and each sample text is calculated respectively, and a target template is selected from the plurality of candidate text templates according to the distance.

[0371] In some embodiments, the sample text includes text entries of multiple dimensions, and the processor 501 runs the application stored in the memory 502, and can also implement the following functions:

[0372] The text items and the sample text in each dimension are obtained, and the distances of the text items in each dimension are calculated respectively; the distance between the text item and the sample text is calculated according to the preset penalty factor and the distances of the text items in each dimension.

[0373] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0374] According to the pixel information of the saliency map, the balanced energy value when the text plate is located at each preset candidate position in the background image is calculated; according to the balanced energy value, the target position of the text plate in the background image is screened out from each candidate position.

[0375] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0376] The pixel information of the text block when it is located at the candidate position is obtained; the energy parameter is calculated according to the saliency map and the pixel information of the text block; and the balanced energy value of the text block when it is located at the candidate position is calculated according to the preset weight and energy parameter.

[0377] In some embodiments, the energy parameter includes a conflict parameter, and the processor 501 runs an application stored in the memory 502, and can also implement the following functions:

[0378] According to the pixel information of the salient map and the text plate, the overlapping pixels of the salient map and the text plate are counted; according to the overlapping pixels, the conflict value of the salient map and the text plate is calculated to obtain the conflict parameter.

[0379] In some embodiments, the energy parameter includes a blank parameter, and the processor 501 runs an application stored in the memory 502, and can also implement the following functions:

[0380] According to the pixel information of the salient map and the text block, the blank pixels in the salient map and the text block are counted; the blank rate is calculated according to the blank pixels to obtain the blank parameter.

[0381] In some embodiments, the energy parameter packet alignment parameters, the processor 501 runs the application stored in the memory 502, and can also implement the following functions:

[0382] According to the pixel information of the salient map and the text plate, the deviation value of the salient map and the text plate in the preset direction is calculated; according to the preset coefficient and the deviation value corresponding to the preset direction, the alignment parameter is calculated.

[0383] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0384] The center position information of the background image and the text block is obtained, and the center parameter is calculated based on the center position information; the balance energy value when the text block is located at the candidate position is calculated based on the preset weight, energy parameter and center parameter.

[0385] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0386] Obtain subject information, and filter out candidate text templates from a preset template library based on the subject information.

[0387] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0388] Obtain keyword information, and select multiple candidate background images from a preset background library based on the keyword information; obtain description texts corresponding to each candidate background image, and calculate the similarity between each description text and the text item; and select a background image from multiple candidate background images based on the similarity.

[0389] In some embodiments, the processor 501 runs the application program stored in the memory 502 and can also implement the following functions:

[0390] Obtaining the performance value corresponding to each candidate background image; calculating the total score of each candidate background image according to the performance value and the similarity; and selecting a background image from multiple candidate background images according to the total score of each candidate background image.

[0391] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0392] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0393] To this end, an embodiment of the present invention provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image generation methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0394] The method comprises the following steps: obtaining an input text item, filtering out a target template from preset candidate text templates according to the text item; typeset the text item according to the target template to generate a text block; obtaining a background image, extracting a saliency map of the background image; obtaining pixel information of the saliency map, and determining a target position of the text block in the background image according to the pixel information of the saliency map; typeset the background image and the text block according to the target position to generate a target image.

[0395] In some embodiments, there are multiple candidate text templates, and the instruction may further perform the following steps:

[0396] A plurality of sample texts included in each candidate text template is obtained; a distance between the text item and each sample text is calculated respectively, and a target template is selected from the plurality of candidate text templates according to the distance.

[0397] In some embodiments, the sample text includes text entries of multiple dimensions, and the instruction may further execute the following steps:

[0398] The text items and the sample text in each dimension are obtained, and the distances of the text items in each dimension are calculated respectively; the distance between the text item and the sample text is calculated according to the preset penalty factor and the distances of the text items in each dimension.

[0399] In some embodiments, the instruction may further perform the following steps:

[0400] According to the pixel information of the saliency map, the balanced energy value when the text plate is located at each preset candidate position in the background image is calculated; according to the balanced energy value, the target position of the text plate in the background image is screened out from each candidate position.

[0401] In some embodiments, the instruction may further perform the following steps:

[0402] The pixel information of the text block when it is located at the candidate position is obtained; the energy parameter is calculated according to the saliency map and the pixel information of the text block; and the balanced energy value of the text block when it is located at the candidate position is calculated according to the preset weight and energy parameter.

[0403] In some embodiments, the energy parameter includes a conflict parameter, and the instruction may further perform the following steps:

[0404] According to the pixel information of the salient map and the text plate, the overlapping pixels of the salient map and the text plate are counted; according to the overlapping pixels, the conflict value of the salient map and the text plate is calculated to obtain the conflict parameter.

[0405] In some embodiments, the energy parameter includes a blank parameter, and the instruction may further perform the following steps:

[0406] According to the pixel information of the salient map and the text block, the blank pixels in the salient map and the text block are counted; the blank rate is calculated according to the blank pixels to obtain the blank parameter.

[0407] In some embodiments, the energy parameter packet alignment parameter, the instruction may further perform the following steps:

[0408] According to the pixel information of the salient map and the text plate, the deviation value of the salient map and the text plate in the preset direction is calculated; according to the preset coefficient and the deviation value corresponding to the preset direction, the alignment parameter is calculated.

[0409] In some embodiments, the instruction may further perform the following steps:

[0410] The center position information of the background image and the text block is obtained, and the center parameter is calculated based on the center position information; the balance energy value when the text block is located at the candidate position is calculated based on the preset weight, energy parameter and center parameter.

[0411] In some embodiments, the instruction may further perform the following steps:

[0412] Obtain subject information, and filter out candidate text templates from a preset template library based on the subject information.

[0413] In some embodiments, the instruction may further perform the following steps:

[0414] Obtain keyword information, and select multiple candidate background images from a preset background library based on the keyword information; obtain description texts corresponding to each candidate background image, and calculate the similarity between each description text and the text item; and select a background image from multiple candidate background images based on the similarity.

[0415] In some embodiments, the instruction may further perform the following steps:

[0416] Obtaining the performance value corresponding to each candidate background image; calculating the total score of each candidate background image according to the performance value and the similarity; and selecting a background image from multiple candidate background images according to the total score of each candidate background image.

[0417] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0418] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0419] Since the instructions stored in the storage medium can execute the steps in any image generation method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any image generation method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0420] The above is a detailed introduction to an image generation method, device and storage medium provided by an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image generation method, characterized in that: include: Obtaining an input text item, and filtering out a target template from preset candidate text templates according to the text item; Typesetting the text item according to the target template to generate a text section; Acquire a background image, and extract a saliency map of the background image; Acquiring pixel information of the saliency map, and calculating, according to the pixel information of the saliency map, a balanced energy value when the text block is located at each preset candidate position in the background image; According to the balance energy value, selecting a target position of the text block in the background image from among the preset candidate positions; According to the target position, the background image and text panels are typeset to generate a target image.

2. The image generation method according to claim 1, characterized in that: There are multiple candidate text templates, and selecting a target template from the preset candidate text templates according to the text item includes: Obtain multiple sample texts contained in each candidate text template; The distances between the text item and each sample text are calculated respectively, and a target template is selected from the plurality of candidate text templates according to the distances, wherein the distances are used to describe the degree of similarity between the text item and the sample text.

3. The image generation method according to claim 2, characterized in that: The sample text includes text items of multiple dimensions, and calculating the distance between the text items and the sample text includes: Obtaining text items of the text item and the sample text in each dimension, and calculating the distance between the text items in each dimension respectively; The distance between the text item and the sample text is calculated based on a preset penalty factor and the distance between the text items in each dimension.

4. The image generation method according to claim 1, wherein: Calculating the balance energy value of the text block when it is located at each preset candidate position in the background image according to the pixel information of the saliency map includes: Acquire pixel information when the text block is located at each of the preset candidate positions; Calculating energy parameters according to the pixel information of the saliency map and the text plate; The balanced energy value when the text block is located at each of the preset candidate positions is calculated based on the preset weight and the energy parameter.

5. The image generation method according to claim 4, characterized in that: The energy parameter includes a conflict parameter, and the energy parameter is calculated based on the pixel information of the saliency map and the text block, including: According to the pixel information of the saliency map and the text block, counting the overlapping pixels of the saliency map and the text block; According to the overlapping pixels, the conflict value between the saliency map and the text plate is calculated to obtain a conflict parameter.

6. The image generation method according to claim 4, characterized in that: The energy parameter includes a blanking parameter, and the energy parameter is calculated based on the pixel information of the saliency map and the text block, including: According to the pixel information of the salient map and the text block, counting the blank pixels in the salient map and the text block; The blanking rate is calculated according to the blanking pixels to obtain a blanking parameter.

7. The image generation method according to claim 4, characterized in that: The energy parameter package alignment parameter, the energy parameter is calculated based on the pixel information of the saliency map and the text plate, including: Calculating a deviation value between the saliency map and the text plate in a preset direction according to pixel information of the saliency map and the text plate; An alignment parameter is calculated according to a preset coefficient corresponding to the preset direction and the deviation value.

8. The image generation method according to claim 4, characterized in that: The step of calculating the equilibrium energy value of the text block when it is located at each of the preset candidate positions according to the preset weight and the energy parameter includes: Obtaining the center position information of the background image and the text plate, and calculating the center parameter according to the center position information; The balanced energy value when the text block is located at each of the preset candidate positions is calculated based on the preset weight, the energy parameter and the center parameter.

9. The image generation method according to claim 1, wherein: The step of selecting a target template from preset candidate text templates according to the text item may include: The subject information is obtained, and candidate text templates are screened out from a preset template library according to the subject information.

10. The image generation method according to claim 1, wherein: The obtaining of the background image comprises: Acquire keyword information, and select multiple candidate background images from a preset background library according to the keyword information; Obtaining description texts corresponding to each candidate background image, and calculating the similarity between each description text and the text item; A background image is selected from the plurality of candidate background images according to the similarity.

11. The image generation method according to claim 10, characterized in that: The selecting a background image from the plurality of candidate background images according to the similarity comprises: Obtain the performance value corresponding to each candidate background image; Calculate the total score of each candidate background image according to the performance value and similarity; A background picture is selected from the plurality of candidate background pictures according to the total score of each candidate background picture.

12. An image generating device, characterized in that: include: A template unit, used to obtain an input text item and select a target template from preset candidate text templates according to the text item; A section unit, used for typeset the text item according to the target template to generate a text section; A saliency map unit, used to obtain a background image and extract a saliency map of the background image; a position unit, configured to obtain pixel information of the saliency map, and calculate, based on the pixel information of the saliency map, a balanced energy value when the text block is located at each preset candidate position in the background image; and selecting a target position of the text block in the background image from among the preset candidate positions according to the balance energy value; The image unit is used to typeset the background image and text panels according to the target position to generate a target image.

13. A computer-readable storage medium, characterized in that: The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the image generating method according to any one of claims 1 to 11.

14. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program in the memory to implement the steps in the image generating method according to any one of claims 1 to 11.

Images