Text image generation method and related device

Abstract

The invention discloses a text image generation method and a related device. The method comprises the steps of obtaining a to-be-processed text and a background image; inputting the visual feature of the background image and the attribute feature of each candidate font in the plurality of candidate fonts into a first language model for style matching, and determining a target font matched with the background image in style from the plurality of candidate fonts; rendering the to-be-processed text in the background image according to the target font to generate a text image, wherein the text image comprises the background image and the to-be-processed text of the target font; according to the mode, the background image is matched in style based on the target font, and the to-be-processed text is rendered in the background image with the target font to obtain the text image, so that the to-be-processed text of the target font in the text image and the background image are more harmonious and beautiful, the visual effect of the text image is enhanced, and the generation effect of the text image is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text image generation method and related devices. Background Art

[0002] With the rapid development of computer technology, in the production scenario of text images, it is necessary to automatically generate text images based on the text to be processed and the background image to meet the scenario requirements of efficiently producing text images.

[0003] In the related art, a text image generation method refers to: for a text to be processed and a background image, selecting a font to be rendered from an existing rendering template, and rendering the text to be processed in the background image according to the font to be rendered to generate a text image.

[0004] However, in the above method, the font to be rendered is selected from the existing rendering template, and there is a situation where the font to be rendered does not match the style of the background image, resulting in a lack of harmony and beauty between the text to be processed and the background image in the generated text image, resulting in poor text image generation effect. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a text image generation method and related devices, which render the text to be processed in the background image with a target font that matches the background image to obtain a text image, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, enhancing the visual effect of the text image, thereby improving the generation effect of the text image.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a text image generation method, the method comprising:

[0008] Get the text and background image to be processed;

[0009] Performing style matching on the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts by using a first language model, and determining a target font from the plurality of candidate fonts; the target font matches the background image in style;

[0010] The text to be processed is rendered in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

[0011] On the other hand, an embodiment of the present application provides a text image generation device, the device comprising: an acquisition unit, used to acquire a text to be processed and a background image;

[0012] a matching unit, configured to perform style matching on the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts through a first language model, and determine a target font from the plurality of candidate fonts; the target font matches the background image in style;

[0013] A rendering unit is used to render the text to be processed in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

[0014] On the other hand, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0015] The memory is used to store a computer program and transmit the computer program to the processor;

[0016] The processor is configured to execute the method described in any one of the preceding aspects according to instructions in the computer program.

[0017] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program runs on a computer device, the computer device executes the method described in any one of the above aspects.

[0018] On the other hand, an embodiment of the present application provides a computer program product, including a computer program, which, when executed on a computer device, enables the computer device to execute the method described in any one of the aforementioned aspects.

[0019] It can be seen from the above technical solution that, first, the text to be processed and the background image are obtained; the visual features of the background image and the attribute features of each candidate font in multiple candidate fonts are input into the first language model for style matching, and the target font that matches the background image in style is determined from the multiple candidate fonts; this method is aimed at the case of rendering the text to be processed in the background image to obtain the text image, considering the visual information of the background image represented by the visual features of the background image, and considering the attribute information of each candidate font represented by the attribute features of each candidate font in multiple candidate fonts, through the powerful semantic understanding ability of the first language model, it is possible to select the candidate font that matches the background image in style from multiple candidate fonts as the target font. Then, the text to be processed is rendered in the background image according to the target font to generate a text image, so that the text image includes the background image and the text to be processed in the target font; based on the target font matching the background image in style, the method renders the text to be processed in the background image with the target font to obtain the text image, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, enhancing the visual effect of the text image, thereby improving the generation effect of the text image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technical members in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A system schematic diagram of a text image generation method provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of a text image generation method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of the structure of a significance analysis model provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a saliency image provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a text image provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of another text image provided in an embodiment of the present application;

[0027] Figure 7A schematic diagram of another text image provided in an embodiment of the present application;

[0028] Figure 8 A flowchart of another text image generation method provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the overall process of a text image generation method provided in an embodiment of the present application;

[0030] Fig.10 A structural diagram of a text image generating device provided in an embodiment of the present application;

[0031] Fig.11 A structural diagram of a server provided in an embodiment of the present application;

[0032] Fig.12 A structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0034] At present, the text image generation method is to select the to-be-processed font from the existing rendering template based on the to-be-processed text and the background image, and render the to-be-processed text in the background image according to the to-be-processed font to generate the text image. However, the to-be-processed font selected from the existing rendering template may not match the background image in style, resulting in the to-be-processed text and the background image in the generated text image being not harmonious and beautiful, and there is a problem of poor text image generation effect.

[0035] The embodiment of the present application provides a text image generation method, which comprises obtaining a text to be processed and a background image; inputting the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts into a first language model for style matching, and determining a target font that matches the background image in style from the plurality of candidate fonts; rendering the text to be processed in the background image according to the target font to generate a text image, so that the text image includes the background image and the text to be processed in the target font; and matching the background image in style based on the target font, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, thereby enhancing the visual effect of the text image and thus improving the generation effect of the text image.

[0036] Next, the system architecture of the text image generation method will be introduced. Figure 1 , Figure 1 A system schematic diagram of a text image generation method provided in an embodiment of the present application, the system includes a computer device 100, and the computer device 100 is used to execute the text image generation method.

[0037] The computer device 100 obtains the text to be processed and the background image, wherein the text to be processed is the text to be rendered and generated, and the background image is the image to be rendered and generated.

[0038] As an example, the computer device 100 obtains the text to be processed “text: three cute puppies” and the background image “image: three cute puppies image”.

[0039] The computer device 100 performs style matching on the visual features of the background image and the attribute features of each candidate font in the plurality of candidate fonts through the first language model, and determines a target font from the plurality of candidate fonts; the target font matches the background image in style. The first language model is a model for performing style matching on the background image and the plurality of candidate fonts to determine the target font that best matches the style of the background image. The visual features of the background image represent the visual information of the background image, and the attribute features of each candidate font represent the attribute information of each candidate font.

[0040] As an example, based on the above example, the first language model is a large language model (LLM), and multiple candidate fonts include fonts such as "Kaiti", "Songti" and "Bold". The computer device 100 uses the large language model to style match the attribute features of each font, such as the visual features of "Kaiti", the attribute features of "Songti" and the attribute features of "Bold", and determines that the target font is "Kaiti" from the fonts such as "Kaiti", "Songti" and "Bold". "Kaiti" matches "Image: Three Cute Puppy Images" in style.

[0041] The computer device 100 renders the text to be processed in the background image according to the target font to generate a text image; the text image includes the background image and the text to be processed in the target font.

[0042] As an example, based on the above example, the computer device 100 renders and generates the "text: three cute puppies" in the "image: three cute puppies image" according to the "regular font", and obtains the text image of "text image: three cute puppies image with three cute puppies text"; the "text image: three cute puppies image with three cute puppies text" includes the "image: three cute puppies image" and the "text: three cute puppies" in "regular font".

[0043] That is to say, in the case of rendering the text to be processed in the background image to obtain a text image, the visual information of the background image represented by the visual features of the background image is considered, and the attribute information of each candidate font represented by the attribute features of each candidate font in multiple candidate fonts is considered. Through the powerful semantic understanding ability of the first language model, a candidate font that matches the background image in style can be selected from multiple candidate fonts as the target font; based on the target font matching the background image in style, the text to be processed is rendered in the background image with the target font, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, thereby enhancing the visual effect of the text image and thus improving the generation effect of the text image.

[0044] It should be noted that in the embodiments of the present application, the computer device may be a server or a terminal, and the method provided in the embodiments of the present application may be executed by the terminal or the server alone, or by the terminal and the server in cooperation. Figure 1 The corresponding embodiments are similar, mainly replacing the computer device with a terminal or a server. In addition, when the method provided in the embodiment of the present application is executed by the terminal and the server in cooperation, the steps that need to be reflected on the front-end interface can be executed by the terminal, and some steps that require background calculation and do not need to be reflected on the front-end interface can be executed by the server.

[0045] Among them, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a vehicle-mounted terminal, an extended reality device or an aircraft, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. For example, the terminal and the server can be connected via a network, which can be a wired or wireless network.

[0046] Next, the text image generation method provided in the embodiment of the present application will be introduced in detail by taking a computer device executing the method provided in the embodiment of the present application as an example and combining with the accompanying drawings.

[0047] See also Figure 2 , Figure 2 A flowchart of a text image generation method provided in an embodiment of the present application, the method comprising:

[0048] S201: Acquire the text to be processed and the background image.

[0049] In the embodiment of the present application, considering the situation of generating a text image from text and image, it is necessary to clarify the text to be processed and the background image used to generate the text image; based on this, it is first necessary to obtain the text to be processed and the background image.

[0050] The text to be processed refers to the text required for generating the text image; and the background image refers to the image required for generating the text image.

[0051] As an example of S201, the text to be processed is "text: three cute puppies", the background image is "image: three cute puppies image", and the computer device obtains "text: three cute puppies" and "image: three cute puppies image".

[0052] The step S201 acquires the text to be processed and the background image to provide text and image for generating a text image, thereby laying a foundation for the subsequent generation of the text image.

[0053] S202: Performing style matching on visual features of the background image and attribute features of each candidate font among multiple candidate fonts through a first language model, and determining a target font from the multiple candidate fonts; the target font matches the background image in style.

[0054] In the related art, based on the text to be processed and the background image, a font to be rendered is selected from an existing rendering template, and the text to be processed is rendered in the background image according to the font to be rendered to generate a text image. However, after research, it is found that the font to be rendered selected from the existing rendering template may not match the background image in style, resulting in the lack of harmony and beauty between the text to be processed and the background image in the generated text image, and there is a problem of poor text image generation effect.

[0055] Therefore, in an embodiment of the present application, in order to solve the above problem, it is necessary to consider the matching between the visual information of the background image and the attribute information of each candidate font based on the background image and multiple candidate fonts, and select a target font that matches the background image in style from multiple candidate fonts; based on this, after executing the above S201, the computer device obtains the text to be processed and the background image, and inputs the visual features of the background image and the attribute features of each candidate font in the multiple candidate fonts into the first language model for style matching, and determines the target font that matches the background image in style from the multiple candidate fonts.

[0056] The first language model refers to a model for determining a font that stylistically matches the image from multiple fonts based on the visual information of the image and the attribute information of each of the multiple fonts. The visual features of the background image represent the visual information of the background image; the attribute features of the candidate font represent the attribute information of the candidate font, and the attribute information of the candidate font may include the font shape, thickness, and spacing.

[0057] As a possible implementation method, S202 can be: performing style matching of the visual features of the background image and the attribute features of each candidate font in a plurality of candidate fonts through a first language model to obtain a matching degree between the background image and each candidate font; sorting the plurality of candidate fonts from large to small according to the matching degree between the background image and each candidate font to obtain a font sorting result; and determining a target font according to the first candidate font in the font sorting result.

[0058] In S202, for the case of rendering the text to be processed in the background image to obtain a text image, visual information of the background image represented by visual features of the background image is considered, and attribute information of each candidate font represented by attribute features of each candidate font in multiple candidate fonts is considered. Through the powerful semantic understanding ability of the first language model, a candidate font that matches the background image in style can be selected from multiple candidate fonts as the target font, thereby providing a font basis for a more harmonious and beautiful relationship between the text to be processed and the background image in a subsequently generated text image.

[0059] As an example of S202, based on the above S201 example, the first language model is a large language model, and multiple candidate fonts include "Kaiti", "Songti" and "Bold". The computer device inputs the attribute features of each font, such as the visual features of "Image: Three Cute Puppy Images", the attribute features of "Kaiti", the attribute features of "Songti" and the attribute features of "Bold", into the large language model for style matching, and determines that the target font that matches "Image: Three Cute Puppy Images" in style from among the fonts such as "Kaiti", "Songti" and "Bold" is "Kaiti".

[0060] S203: Rendering the text to be processed in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

[0061] In an embodiment of the present application, after executing the above S202, the computer device determines the target font that matches the background image in style from multiple candidate fonts, and renders the text to be processed in the background image according to the target font to generate a text image. The text image can be obtained including the background image and the text to be processed in the target font, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful.

[0062] As a possible implementation manner, S203 may be: performing font conversion on the text to be processed according to the target font to obtain the text to be processed in the target font; rendering the text to be processed in the target font in the background image to obtain a text image.

[0063] The S203 matches the background image in style based on the target font, and renders the text to be processed in the background image with the target font, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, thereby enhancing the visual effect of the text image and thus improving the generation effect of the text image.

[0064] As an example of S203, based on the above S202 example, the computer device renders the "text: three cute puppies" in the "image: three cute puppies image" according to "Kai Ti" to generate a text image of "text image: three cute puppies image with three cute puppies text"; the "text image: three cute puppies image with three cute puppies text" includes the "image: three cute puppies image" and the "text: three cute puppies" in "Kai Ti".

[0065] As can be seen from the above technical solution, the text to be processed and the background image are obtained; the visual features of the background image and the attribute features of each candidate font in multiple candidate fonts are input into the first language model for style matching, and the target font that matches the background image in style is determined from the multiple candidate fonts; for the case of rendering the text to be processed in the background image to obtain a text image, the visual information of the background image represented by the visual features of the background image is considered, and the attribute information of each candidate font represented by the attribute features of each candidate font in multiple candidate fonts is considered, and through the powerful semantic understanding ability of the first language model, the candidate font that matches the background image in style can be selected from multiple candidate fonts as the target font. The text to be processed is rendered in the background image according to the target font to generate a text image, so that the text image includes the background image and the text to be processed in the target font; based on the target font matching the background image in style, the text to be processed is rendered in the background image with the target font to obtain a text image, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, and the visual effect of the text image is enhanced, thereby improving the generation effect of the text image.

[0066] In an embodiment of the present application, in the above S202, in which the first language model is used to perform style matching on the visual features of the background image and the attribute features of each candidate font in multiple candidate fonts, in order to implement the first language model for determining a target font that matches the background image in style from multiple fonts based on the visual information of the background image and the attribute information of each candidate font in multiple candidate fonts, the first language model can be guided by a style matching prompt, so that the first language model considers the matching between the visual information of the background image and the attribute information of each candidate font, and determines the target font that matches the background image in style from multiple candidate fonts. Based on this, the present application provides a possible implementation method, S202 includes the following S2021 (not shown in the figure).

[0067] S2021: Performing style matching of visual features and attribute features of each candidate font according to style matching hints through the first language model, and determining a target font from multiple candidate fonts; the style matching hint is used to guide the first language model to determine a candidate font that stylistically matches the background image from multiple candidate fonts.

[0068] The S2021 can use style matching hints to enable the first language model to accurately guide the first language model to determine a candidate font that matches the style of the background image from multiple candidate fonts as the target font when performing style matching on the visual features of the background image and the attribute features of each candidate font, thereby selecting a target font that is highly consistent with the background image from multiple candidate fonts, enhancing the visual effect of rendering the text to be processed in the target font on the background image, and making the text to be processed in the target font and the background image more harmonious and beautiful.

[0069] As an example, based on the above examples of S201-S203, the style matching prompt is PromptX as shown below.

[0070] “Title: Determine a font that matches the style of a background image from multiple candidate fonts.

[0071] Visual characteristics of the background image:

[0072] 1. Color tone: The background image is dominated by [specific color tone, such as "soft white"], creating [such as "comfortable atmosphere"].

[0073] 2. Elements: The image contains [specific elements, such as “a sitting puppy”], which together form a picture [such as “cute and comfortable”].

[0074] List of candidate fonts and their attribute characteristics:

[0075] 1. Kaiti:

[0076] Font: Kaiti maintains the blunt and running form of Kaishu, with flexible strokes, slightly different thicknesses of horizontal and vertical strokes, and tilted to the upper right. The strokes of dots, horizontal, vertical, downward, downward, upward, and hooks are solid and soft. Kaiti's style is close to handwriting, giving people a friendly and easy-to-read feeling.

[0077] Thickness: The thickness of the strokes in Kai script varies naturally, with slightly different horizontal and vertical thicknesses. But overall, the thickness changes are not as obvious as those in Song script.

[0078] Spacing: The spacing between characters and lines of regular script is usually adjusted according to the needs of writing or typesetting to ensure overall beauty and readability.

[0079] 2. Songti:

[0080] Font: Song is a Chinese font that was developed to adapt to printing. Its font is square, with a large difference in thickness between horizontal and vertical strokes, usually thin horizontal strokes and thick vertical strokes. There are triangular node serif decorations at the end of the strokes, which are called "foots" or "serifs". Song is highly decorative, giving people an impression of elegance, neatness, solemnity and generosity.

[0081] Thickness: The thickness of the strokes of Song font varies significantly, with horizontal strokes thin and vertical strokes thick. This design makes Song font have a better visual effect when printed and can clearly present the text content.

[0082] Spacing: The spacing between characters and lines of Song font also needs to be adjusted according to the typesetting needs. In books, magazines, newspapers and other printed materials, Song font is often used for text typesetting, so its spacing setting needs to take into account the reader's reading experience.

[0083] 3. Bold:

[0084] Font: Boldface is also called square or Gothic, and is a sans serif font. Its font is neat, with horizontal and vertical strokes, all the strokes are of the same thickness, and there is no serif decoration. Boldface fonts are simple and elegant, with a sense of fashion and modernity.

[0085] Thickness: The strokes of boldface are of consistent thickness, without obvious thickness changes. This design makes boldface more visually stable and powerful. Boldface can be divided into bold, large, medium, thin, round-headed boldface and other types to meet different typesetting needs.

[0086] Spacing: The spacing between characters and lines of boldface also needs to be adjusted according to the needs of typesetting. Because its strokes are of uniform thickness and there are no serif decorations, boldface needs to pay special attention to the setting of character spacing and line spacing when typesetting to avoid a visual effect that is too crowded or loose.

[0087] …

[0088] Analysis requirements:

[0089] Carefully analyze the tone, elements, and overall atmosphere of the background image, and think about which font attributes can match the background image; consider the font shape, weight, spacing and other attribute characteristics, and determine whether the attribute characteristics of the candidate font match the visual characteristics of the background image. "

[0090] Then S2021 can be: through the large language model, according to PromptX, the visual features of "Image: Three Cute Puppy Images", the attribute features of "Kai Ti", the attribute features of "Song Ti", and the attribute features of "Bold" and other attribute features of each font are style matched, and the target font that matches "Image: Three Cute Puppy Images" in style is determined to be "Kai Ti" from among the fonts such as "Kai Ti", "Song Ti" and "Bold".

[0091] In the embodiment of the present application, in the above S203, in which the text to be processed is rendered in the background image according to the target font to generate a text image, considering that when rendering the text to be processed in the target font in the background image, in order to avoid the text to be processed in the target font with a certain aspect ratio from covering a large area of ​​the key area of ​​the background image, and the key area of ​​the background image can be reflected by the saliency image of the background image, it is also necessary to determine the target text area in which the text to be processed in the target font is rendered in the background image based on the aspect ratio of the text to be processed in the target font and the saliency image of the background image. Therefore, on the basis of obtaining the aspect ratio of the text to be processed in the target font and the saliency image of the background image, the saliency image of the background image and the aspect ratio of the text to be processed in the target font are input into the text area generation model, and multiple text areas with less occlusion or unoccluded key areas of the background image that meet the aspect ratio are generated, thereby outputting multiple first text areas; a first text area that does not cover the key area of ​​the background image is selected from the multiple first text areas as the target text area; thereby rendering the text to be processed in the text area of ​​the background image according to the target font to generate a text image. Based on this, the present application provides a possible implementation method. The text image generation method further includes the following S1-S2 (not shown in the figure) after the above S202 and before the above 203. Correspondingly, S203 includes the following S2031 (not shown in the figure).

[0092] S1: Generate text regions for the background image using a text region generation model according to the aspect ratio of the text to be processed in the target font and the saliency image of the background image, to obtain a plurality of first text regions.

[0093] S2: Determine a target text region from multiple first text regions.

[0094] S2031: In the target text area of ​​the background image, the text to be processed is rendered according to the target font to obtain a text image.

[0095] Among them, the text area generation model refers to a model that generates multiple text areas based on the aspect ratio of the text and the saliency image of the image; the aspect ratio of the text to be processed in the target font refers to the fixed ratio of the length and width of the text area of ​​the text to be processed in the target font; the saliency image of the background image refers to the image of the pixel saliency value of each pixel in the background image; the pixel saliency value of a pixel reflects the criticality of the pixel in the background image; the larger the pixel saliency value of a pixel, the higher the criticality of the pixel in the background image, and the greater the probability that the pixel belongs to the critical area of ​​the background image.

[0096] S1-S2 and S2031 input the aspect ratio of the text to be processed in the target font and the saliency image of the background image into the text area generation model, and on the basis that the aspect ratio reflects the fixed ratio of the length and width of the text area of ​​the text to be processed in the target font and the saliency image reflects the criticality of each pixel in the background image, can generate multiple first text areas in the background image that render the text to be processed in the target font and less occlude or do not occlude the key area of ​​the background image. Since the first text area of ​​the background image and the key area of ​​the background image overlap less or do not overlap, the target text area determined from the multiple first text areas can also less occlude or do not occlude the key area of ​​the background image, thereby achieving that when the text to be processed in the target font is rendered in the target text area of ​​the background image, the text to be processed in the target font does not occlude the key area of ​​the background image on a large scale.

[0097] As an example, based on the above-mentioned examples of S201-S203, the text region generation model is the basic text box generation model (SmartText), and the salient image of “image: three cute puppies image” is “salient image: three cute puppies salient image”, then S1-S2 can be: SmartText is used to generate text regions for “image: three cute puppies image” according to the aspect ratio of “text: three cute puppies” in “Kai Ti” and the “salient image: three cute puppies salient image” of “image: three cute puppies image”, to obtain a plurality of first text regions A. i , i = 1, 2, ..., n, n is a positive integer greater than 2, from multiple A i The target text area is determined as A t Correspondingly, S2031 may be: in A of “image: three cute puppy images” t In the example, the “text: three cute puppies” is rendered and generated according to the “Kai Ti” to obtain the “text image: three cute puppies image with three cute puppies text”.

[0098] In the embodiment of the present application, the more accurate the saliency image of the background image in the above S1 is, the more the target text area generated by executing the above S1 can avoid occluding the key area of ​​the background image. Therefore, the process of obtaining the saliency image of the background image can be: performing saliency analysis on the background image through a saliency analysis model including a segmentation prediction module and a refinement mapping module to obtain the saliency image of the background image; that is, first performing segmentation prediction on the background image through the segmentation prediction module to obtain a segmented image of the background image, and then continuing to refine and map the segmented image through the refinement mapping module to obtain a saliency image of the background image with higher accuracy. Based on this, the present application provides a possible implementation method, and the text image generation method also includes the following S3-S4 (not shown in the figure).

[0099] S3: Perform segmentation prediction on the background image through the segmentation prediction module in the saliency analysis model to obtain a segmented image of the background image.

[0100] S4: The segmented image is refined and mapped by the refinement mapping module in the saliency analysis model to obtain a saliency image.

[0101] The saliency analysis model includes a segmentation prediction module and a refinement mapping module. The segmentation prediction module is used to segment and predict the background image into a segmented image, and the refinement mapping module is used to refine and map the segmented image into a saliency image.

[0102] The S3-S4 first inputs the background image into the segmentation prediction module of the saliency analysis model, and uses the segmentation prediction module to perform segmentation prediction on the background image, so as to obtain a relatively accurate segmented image of the background image; then inputs the segmented image of the background image into the refinement mapping module of the saliency analysis model, and uses the refinement mapping module to continue to perform refinement mapping on the segmented image, so as to finally obtain a more accurate saliency image of the background image.

[0103] As an example, based on the above example of S1-S2, the saliency analysis model is BASNet, and BASNet includes a segmentation prediction module En-De and a refinement mapping module RRM, then S3-S4 can be: using En-De in BASNet to perform segmentation prediction on the "image: three cute puppy image", and obtaining the segmented image of the "image: three cute puppy image" as the "segmented image: three cute puppy segmented image"; using RRM in BASNet to perform refinement mapping on the "segmented image: three cute puppy segmented image", a "saliency image: three cute puppy saliency image" is obtained.

[0104] See also Figure 3 As shown, Figure 3 A structural diagram of a significance analysis model provided in an embodiment of the present application is illustrated. Figure 3 In the present invention, the saliency analysis model includes a segmentation prediction module En-De and a refinement mapping module RRM. The background image is input into En-De to obtain a segmented image, which is then input into the refinement mapping module to obtain a saliency image.

[0105] See also Figure 4 As shown, Figure 4 A schematic diagram of a saliency image provided in an embodiment of the present application is illustrated. Figure 4 The area surrounded by the black dotted line is the area with a higher significance value, that is, the key area in the background image is the area where the three puppies are located.

[0106] In an embodiment of the present application, in the process of determining the target text area from multiple first text areas in the above S2, in order to avoid the to-be-processed text of the target font from obstructing the key area of ​​the background image during text rendering, it is considered that the larger the pixel saliency value of the pixel in the saliency image, the higher the criticality of the pixel in the background image, and the greater the probability that the pixel in the background image belongs to the key area of ​​the background image; therefore, the relevant pixel saliency value of each first text area can be determined through the saliency image, so as to realize the determination of the target text area that does not obstruct the key area of ​​the background image from the first text area.

[0107] One implementation method is: first determine multiple pixel significance values ​​of each first text region through a significance image, and for multiple first text regions, delete a preset number of first text regions with pixel significance values ​​greater than the first significance value to obtain multiple second text regions; then determine the average significance value of each second text region through a significance image, and for multiple second text regions, select the second text region with an average significance value less than the second significance value as a candidate text region; so as to determine the target text region based on the candidate text region. Based on this, the present application provides a possible implementation method, S2 includes the following S2a-S2e (not shown in the figure).

[0108] S2a: Determine a plurality of pixel saliency values ​​of each first text region according to the saliency image.

[0109] S2b: According to the multiple pixel significance values ​​of each first text region, a preset number of first text regions having pixel significance values ​​greater than the first significance value are deleted from the multiple first text regions to obtain multiple second text regions.

[0110] S2c: Determine the average saliency value of each second text region according to the saliency image.

[0111] S2d: According to the average significance value of each second text region, a second text region having an average significance value less than the second significance value is selected from the plurality of second text regions to obtain a candidate text region.

[0112] S2e: Determine the target text region based on the candidate text regions.

[0113] The average significance value of each first text region is calculated based on the pixel significance values ​​of a plurality of pixels included in each first text region.

[0114] S2a-S2e can determine multiple pixel significance values ​​in each first text region through the significance image. When the number of pixel significance values ​​greater than the first significance value among the multiple pixel significance values ​​included in the first text region reaches a preset number, it indicates that the first text region blocks the key region of the background image. Then, the preset number of first text regions with pixel significance values ​​greater than the first significance value are deleted from the multiple first text regions, thereby obtaining multiple second text regions. The pixel significance values ​​of multiple pixels in each second text region are continuously determined through the significance image to calculate the average significance value. When the average significance value of the second text region is less than the second significance value, it indicates that the second text region does not block the key region of the background image. Then, the second text region with the average significance value less than the second significance value is selected from the multiple second text regions as the candidate text region. The target text region that does not block the key region of the background image can be accurately determined through the candidate text region. That is to say, the target text area is to first delete the first text area that may obstruct the key area of ​​the background image based on multiple pixel significance values, and then retain the second text area of ​​the key area that does not obstruct the background image based on the average significance value, so as to screen out the candidate text area, and finally determine the target text area of ​​the key area that does not obstruct the background image based on the candidate text area, so as to further avoid the text to be processed from obstructing the key area of ​​the background image.

[0115] Another implementation method is: first determine the average saliency value of each first text area through a saliency image, select the first text area with an average saliency value less than the second saliency value for multiple first text areas, obtain multiple third text areas, delete a preset number of third text areas with pixel saliency values ​​greater than the first saliency value for multiple third text areas, obtain multiple third text areas, and use the third text areas as candidate text areas; so as to determine the target text area based on the candidate text areas. Based on this, the present application provides a possible implementation method, S2 includes the following S2f-S2j (not shown in the figure).

[0116] S2f: Determine the average saliency value of each first text region according to the saliency image.

[0117] S2g: According to the average significance value of each first text region, first text regions having an average significance value less than the second significance value are selected from the plurality of first text regions to obtain a plurality of third text regions.

[0118] S2h: Determine multiple pixel saliency values ​​of each third text region according to the saliency image.

[0119] S2i: According to the multiple pixel significance values ​​of each third text region, a preset number of third text regions having pixel significance values ​​greater than the first significance value are deleted from the multiple third text regions to obtain candidate text regions.

[0120] S2j: Determine the target text region based on the candidate text regions.

[0121] The average significance value of each first text region is calculated based on the pixel significance values ​​of a plurality of pixels included in each first text region.

[0122] S2f-S2j can determine multiple pixel significance values ​​in each first text area through the significance image, and calculate the average significance value of each first text area according to the multiple pixel significance values. When the average significance value of the first text area is less than the second significance value, it indicates that the first text area does not block the key area of ​​the background image. Then, first text areas with average significance values ​​less than the second significance value are selected from the multiple first text areas to obtain multiple third text areas. The pixel significance values ​​of multiple pixels in each third text area are continuously determined through the significance image. When the number of pixel significance values ​​greater than the first significance value among the multiple pixel significance values ​​included in the third text area reaches a preset number, it indicates that the third text area blocks the key area of ​​the background image. Then, a preset number of third text areas with pixel significance values ​​greater than the first significance value are deleted from the multiple third text areas, thereby obtaining candidate text areas. The target text area that does not block the key area of ​​the background image can be accurately determined through the candidate text areas. That is to say, the target text area is firstly to retain the third text area of ​​the key area of ​​the unobstructed background image based on the average significance value, and then to delete the third text area of ​​the key area of ​​the occluded background image based on multiple pixel significance values, so as to screen out the candidate text area, and finally to determine the target text area of ​​the key area of ​​the unobstructed background image according to the candidate text area, so as to further avoid the text to be processed from occluding the key area of ​​the background image.

[0123] In an embodiment of the present application, when the above S2e and S2j determine the target text area through the candidate text area, it is further considered that when the candidate text area includes multiple candidate text areas, the position aesthetics of each candidate text area in the background image affects the position aesthetics of the target font's to-be-processed text rendering in the background image; therefore, each of the multiple candidate text areas can be input into the aesthetic prediction model for aesthetic prediction to predict the position aesthetics of each candidate text area in the background image, and the aesthetic prediction data of each candidate text area is output. Through the aesthetic prediction data of each candidate text area, the target text area can be determined from the multiple candidate text areas, so as to determine the target text area with higher position aesthetics. Based on this, the present application provides a possible implementation method, S2e and S2j include the following S5-S6 (not shown in the figure).

[0124] S5: If the candidate text region includes multiple candidate text regions, perform aesthetic prediction on each candidate text region of the background image using an aesthetic prediction model to obtain aesthetic prediction data for each candidate text region.

[0125] S6: Determine a target text region from multiple candidate text regions according to the aesthetic prediction data of each candidate text region.

[0126] The aesthetic prediction model refers to a model for predicting the aesthetic level of the position of a region in an image; and the aesthetic prediction data refers to data reflecting the aesthetic level of the position of a candidate text region.

[0127] In S5-S6, the aesthetic prediction model is used to perform aesthetic prediction on multiple candidate text regions of the background image, and the aesthetic prediction data of each candidate text region is obtained, that is, the aesthetic score of each candidate text region is obtained, and the target text region is determined from the multiple candidate text regions according to the aesthetic score of each candidate text region. The multiple candidate text regions are further screened by the aesthetic prediction model to obtain the target text region, and the target text region with aesthetics and rationality is obtained, thereby improving the position aesthetics of the text to be processed when it is rendered in the target text region.

[0128] As a possible implementation method, S6 may be: aesthetically sorting multiple candidate text regions from large to small according to the aesthetic prediction data of each candidate text region to obtain an aesthetic sorting result; and determining the target text region according to the first candidate text region in the aesthetic sorting result.

[0129] As an example, based on the above example of 201-S203, S5-S6 can be: if the candidate text region includes multiple candidate text regions, perform aesthetic prediction on each candidate text region of "image: three cute puppy images" through the aesthetic prediction model, and obtain aesthetic prediction data Y for each candidate text region. According to Y of each candidate text region, determine A from the multiple candidate text regions. t .

[0130] In the embodiment of the present application, it is further considered that the text image finally rendered includes a background image and a text to be processed in a target font, and the text to be processed is used to simply and clearly display the content in the background image. In order to make the text to be processed clear and eye-catching in the text image, the target color of the text to be processed can be determined by the background color of the target text area of ​​the text to be processed. In order to determine the background color of the target text area, the background image can be color clustered to obtain the main color of the background image, and then the background color of the target text area can be determined by the main color of the background image. Considering that the greater the contrast between the target color of the text to be processed and the background color of the target text area, the clearer and more eye-catching the text to be processed can be displayed in the target text area, therefore, the target color of the text to be processed can be determined from multiple candidate colors by the contrast between the background color and each candidate color in multiple candidate colors. After determining the target color of the text to be processed, the text to be processed can be rendered in the text area of ​​the background image according to the target font and the target color to generate a text image. Based on this, the present application provides a possible implementation method, and the text image generation method also includes the following S7-S9 (not shown in the figure), and correspondingly, S2031 includes the following S2031a (not shown in the figure).

[0131] S7: Perform color clustering on the background image to obtain the main color of the background image.

[0132] S8: Determine the background color of the target text area according to the main color of the background image.

[0133] S9: Determine a target color of the text to be processed from the multiple candidate colors according to the contrast between the background color and each candidate color in the multiple candidate colors.

[0134] S2031a: In the target text area of ​​the background image, the text to be processed is rendered according to the target font and the target color to obtain a text image.

[0135] The contrast ratio between the background color and each candidate color among the multiple candidate colors refers to the brightness ratio between the brightness of the background color and the brightness of each candidate color.

[0136] S7-S9 and S2031a perform color clustering on the background image to obtain the main color of the background image, and considering that the position of the text to be processed is the target text area, the background color of the target text area is determined according to the main color of the background image, and the target color of the text to be processed is determined from the multiple candidate colors according to the contrast between the background color of the target text area and each candidate color in the multiple candidate colors. In other words, the target color of the text to be processed is determined based on the background color of the target text area, so that the text to be processed can be highlighted when it is set in the target text area, so that the text to be processed is clear and eye-catching in the background image, and the readability and transmission effect of the text to be processed are improved.

[0137] As an example, based on the example of S2031 above, S7-S9 may be: perform color clustering on the “image: three cute puppy images” using the k-means algorithm, and obtain that the main colors of the “image: three cute puppy images” are “white”, “gray”, and “black”; determine A according to the “white”, “gray”, and “black” of the “image: three cute puppy images”. t is "black"; according to the contrast between "black" and each candidate color of "black", "white", "red", etc., the target color of "text: three cute puppies" is determined to be "white" from each candidate color of "black", "white", "red", etc. Correspondingly, S2031a can be: in A of "image: three cute puppies image" t In the example, “text: three cute puppies” is rendered and generated according to “regular font” and “white” to obtain “text image: three cute puppies image with three cute puppies text”.

[0138] See also Figure 5 As shown, Figure 5 A schematic diagram of a text image is shown. Figure 5 The background image is an image of three cute puppies, the text to be processed is three cute puppies, the main colors of the background image are white, gray and black, the target text area is the area where the white dotted box is located, the background color of the target text area is black, and the target color of the text to be processed is white.

[0139] In the embodiment of the present application, it is further considered that the text to be processed includes keywords and non-keywords when describing the content of the background image. In order to distinguish the colors of keywords and non-keywords, a first color can be determined from multiple candidate colors as the color of the keywords of the text to be processed, and a second color can be determined from multiple candidate colors as the color of the non-keywords of the text to be processed, according to the background color of the target text area and the contrast of each candidate color. Then, in the target text area of ​​the background image, the keywords and non-keywords of the text to be processed are rendered and generated according to the target font, the first color and the second color, and finally a text image is obtained, so that the keywords and non-keywords of the text to be processed included in the text image are distinguished by the first color and the second color respectively. Based on this, the present application provides a possible implementation method, and the text image generation method also includes the following S10 (not shown in the figure), S9 includes the following S91 (not shown in the figure), and S2031a includes the following S2031aa (not shown in the figure).

[0140] S10: Obtain keywords to be processed and non-keywords to be processed of the text to be processed.

[0141] S91: Determine, from a plurality of candidate colors, according to the contrast between the background color and each candidate color, a target color including a first color of a keyword to be processed and a second color of a non-keyword to be processed; the first color is different from the second color.

[0142] S2031aa: In the target text area of ​​the background image, the keywords to be processed and the non-keywords to be processed are rendered and generated according to the target font, the first color and the second color to obtain a text image.

[0143] The keywords to be processed are keywords of the text to be processed, and the non-keywords to be processed are non-keywords of the text to be processed.

[0144] The S10, S91 and S2031aa obtain the keywords to be processed and non-keywords to be processed of the text to be processed, and determine the first color of the keywords to be processed and the second color of the non-keywords to be processed from multiple candidate colors according to the contrast between the background color of the target text area and each candidate color, and set the first color and the second color to be different, so as to achieve different colors of the keywords and non-keywords of the text to be processed, and continue to render and generate the keywords and non-keywords of the text to be processed in the target text area of ​​the background image according to the target font, the first color and the second color, and obtain a text image including the keywords of the first color and the non-keywords of the second color, so as to achieve the use of the first color and the second color to distinguish the keywords and non-keywords of the text to be processed in the text image, and visually distinguish the keywords and non-keywords of the text to be processed.

[0145] As an example, based on the above examples of S7-S9 and S2031a, S10, S91 and S2031aa may be: obtaining the keyword to be processed of "text: three cute puppies" as "puppy" and the non-keyword to be processed as "three cute"; determining the target color from multiple candidate colors such as "black", "white", "red", etc., including the first color of "puppy" as "white" and the second color of "three cute" as "red" according to the contrast between "black" and each candidate color in each candidate color such as "black", "white", "red"; in A of "image: three cute puppy images", t In the example, “puppy” and “three cute puppies” are rendered and generated according to “regular font”, “white” and “red” to obtain “text image: three cute puppies image with three cute puppies text”.

[0146] See also Figure 6 As shown, Figure 6 A schematic diagram of another text image is shown. Figure 6 The background image is an image of three cute puppies, the text to be processed is three cute puppies, the keyword to be processed is puppy, the non-keyword to be processed is three cute, the main colors of the background image are white, gray and black, the target text area is the area where the white dotted box is located, the background color of the target text area is black, and the keyword to be processed is Figure 6 The word "puppy" is surrounded by a solid white ellipse. The first color of "puppy" is white. The non-keywords to be processed are Figure 6 “Three lovely ones” are surrounded by the white dotted ellipse. The second color of “Three lovely ones” is red.

[0147] In an embodiment of the present application, when the above S91 determines the first color of the keyword to be processed and the second color of the non-keyword to be processed from multiple candidate colors according to the contrast between the background color and each candidate color, considering the need to highlight the keywords so as to facilitate intuitive and eye-catching key points of the text image, the multiple candidate colors can be sorted from large to small according to the contrast between the background color and each candidate color to obtain a color sorting result, and the first color is determined according to the first candidate color in the color sorting result, and the second color is determined according to the second candidate color in the color sorting result, that is, the candidate color with the largest contrast with the background color is used as the first color of the keyword to be processed, and the candidate color with the second largest contrast with the background color is used as the second color of the non-keyword to be processed. Based on this, the present application provides a possible implementation method, and S91 also includes the following S91a-S91c (not shown in the figure).

[0148] S91a: Sort the multiple candidate colors from large to small according to the contrast between the background color and each candidate color to obtain a color sorting result.

[0149] S91b: Determine a first color according to the first candidate color in the color sorting result.

[0150] S91c: Determine a second color according to the second candidate color in the color sorting result.

[0151] S91a-S91c sorts multiple candidate colors from large to small according to the contrast between the background color and each candidate color to obtain a color sorting result, and determines the first candidate color in the color sorting result as the first color of the keyword of the text to be processed, and determines the second candidate color in the color sorting result as the second color of the non-keyword of the text to be processed, thereby achieving the distinguishing display of the keywords and non-keywords of the text to be processed in the text image and more prominent display of the keywords.

[0152] As an example, based on the examples of S10, S91 and S2031aa above, S91a-S91c can be: according to the contrast between "black" and each candidate color in "black", "white", "red", etc., the contrast of each candidate color in "black", "white", "red", etc. is sorted from large to small, and the color sorting results are "white", "red" and "black". According to the first candidate color in "white", "red" and "black", the first color is determined to be "white". According to the second candidate color in "white", "red" and "black", the second color is determined to be "red".

[0153] In an embodiment of the present application, when the above S10 obtains the keywords to be processed and the non-keywords to be processed of the text to be processed, considering the readability and transmission effect of the keywords to be processed and the non-keywords to be processed, it is necessary to accurately segment the text to be processed. The powerful semantic understanding ability of the second language model can be used for word segmentation processing, that is, the text to be processed is preprocessed by the second language model to obtain multiple segmented words and keywords to be processed of the text to be processed, and the keywords to be processed are deleted from the multiple segmented words to be processed, thereby obtaining the non-keywords to be processed. Based on this, the present application provides a possible implementation method, and S10 also includes the following S1001-S1002 (not shown in the figure).

[0154] S1001: Preprocessing the text to be processed by using a second language model to obtain a plurality of to-be-processed segmented words and to-be-processed keywords of the text to be processed.

[0155] S1002: Deleting keywords to be processed from multiple word segments to be processed to obtain non-keywords to be processed.

[0156] The second language model refers to a model that preprocesses text into multiple word segments and keywords; text preprocessing includes word segmentation processing and keyword extraction processing.

[0157] S1001-S1002 performs word segmentation processing and keyword extraction processing on the text to be processed through the second language model, and obtains multiple word segmentations and keywords to be processed of the text to be processed, wherein the multiple word segmentations include keywords to be processed, and the keywords to be processed in the multiple word segmentations are deleted to obtain non-keywords to be processed. By utilizing the powerful semantic understanding ability of the second language model, multiple word segmentations and keywords to be processed with high accuracy, readability and information density can be obtained. The text to be processed can be processed into multiple word segmentations through the second language model, and the text to be processed can be rendered in the target text area in the form of multiple word segmentations to be processed, so as to prevent the key areas of the background image from being blocked due to the excessive length of the text to be processed.

[0158] As an example, based on the above examples of S10, S91 and S2031aa, the second language model is a large language model, the first language model and the second language model are the same model, and S1001-S1002 can be: input "text: three cute puppies" into the large language model for text preprocessing, and obtain multiple to-be-processed segmented words of "text: three cute puppies" as "three cute" and "puppy" and the to-be-processed keyword as "puppy". Delete "puppy" from "three cute" and "puppy" to obtain the to-be-processed non-keyword as "three cute".

[0159] When the second language model performs text preprocessing, it can use the text preprocessing prompt to perform text preprocessing. The text preprocessing prompt is shown in PromptZ below.

[0160] "Title: Text preprocessing for text to be processed.

[0161] Processing requirements: Given a text to be processed with Chinese characters, it is hoped that the text to be processed will be segmented and keywords will be extracted according to the meaning of the phrases therein, so that the text semantics of the segmented words of the text to be processed after the segmentation processing are as complete as possible, the number of words is as close as possible, and the keywords are the most critical segmented words to be processed after the segmentation processing. The output format can use " / " to represent the segmentation processing, and the parentheses represent the keywords. For example, "I found a durian puff", the output format is "I found / a / (durian puff)", and the keyword is "durian puff". If no segmentation processing is required, the entire text to be processed is considered as a keyword, for example, the output is (seaside)."

[0162] See also Figure 7 As shown, Figure 7 A schematic diagram of another text image is shown. Figure 7The background image is an image of three cute puppies, and the text to be processed is three cute puppies. Text preprocessing is performed on the text to obtain multiple to-be-processed segmented words "three cute / (puppies)", the to-be-processed keyword is "puppies", and the to-be-processed non-keyword is "three cute". The to-be-processed keyword and the to-be-processed non-keyword may not be displayed in the same row of the target text area of ​​the text image.

[0163] In the embodiment of the present application, the multiple candidate fonts are fonts already available in the font library, that is, when rendering the text to be processed according to the target font, the fonts already available in the font library are used for rendering, which can avoid typos in the rendered text image.

[0164] From the above description, it can be seen that another text image generation method provided in the embodiment of the present application includes the following steps: Figure 8 shown.

[0165] S801: Acquire the text to be processed and the background image.

[0166] S802: Preprocessing the text to be processed by using a second language model to obtain a plurality of to-be-processed word segments, to-be-processed keywords, and to-be-processed non-keywords of the text to be processed.

[0167] S803: Perform style matching on the visual features of the background image and the attribute features of each candidate font in the plurality of candidate fonts by using the first language model, and determine a target font from the plurality of candidate fonts.

[0168] S804: Generate text regions for the background image using a text region generation model according to the aspect ratio of the to-be-processed text in the target font and the saliency image of the background image, to obtain a plurality of first text regions.

[0169] S805: Determine a target text region from the plurality of first text regions.

[0170] S806: Perform color clustering on the background image to obtain the main color of the background image.

[0171] S807: Determine the background color of the target text area according to the main color of the background image.

[0172] S808: Determine a target color of the text to be processed from the multiple candidate colors according to the contrast between the background color and each candidate color in the multiple candidate colors.

[0173] S809: In the target text area of ​​the background image, the text to be processed is rendered according to the target font and the target color to obtain a text image.

[0174] See also Fig. 9 As shown, Fig. 9 The overall flow diagram of a text image generation method provided by an embodiment of the present application is illustrated. The text to be processed is obtained as "text: three cute puppies" and the background image is "image: three cute puppy images". The text to be processed is preprocessed by the second language model, that is, word segmentation processing and keyword extraction processing are performed to obtain multiple to-be-processed word segments as "three cute / puppies" and to-be-processed keywords as "puppies". According to the first language model, the visual features of the background image and the attribute features of each candidate font in multiple candidate fonts are style matched to obtain a target font that matches the background image in style, and then continue to generate multiple first text regions of the background image through the text region generation model according to the aspect ratio of the to-be-processed text of the target font and the saliency image of the background image, and the text region is screened from the multiple first text regions according to the saliency value and the aesthetic prediction of the text region is performed according to the aesthetic prediction data, and finally the target text region is determined, and the coordinates of the diagonal vertices of the target text region and the aesthetic prediction data are obtained. The background color of the target text region is obtained, and then the target color of the to-be-processed text is determined according to the background color of the target text region.

[0175] It should be noted that, based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0176] based on Figure 2 Corresponding to the text image generation method provided in the embodiment, the present application embodiment also provides a text image generation device, see Fig.10 , Fig.10 This is a structural diagram of a text image generation device provided in an embodiment of the present application. The text image generation device 1000 includes: an acquisition unit 1010, a matching unit 1020 and a rendering unit 1030;

[0177] An acquisition unit 1010 is used to acquire the text to be processed and the background image;

[0178] A matching unit 1020 is used to perform style matching on the visual features of the background image and the attribute features of each candidate font in the plurality of candidate fonts through a first language model, and determine a target font from the plurality of candidate fonts; the target font matches the background image in style;

[0179] The rendering unit 1030 is used to render the text to be processed in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

[0180] In a possible implementation, the matching unit 1020 is configured to:

[0181] The first language model performs style matching on the visual features and the attribute features of each candidate font according to the style matching hint, and determines the target font from multiple candidate fonts; the style matching hint is used to guide the first language model to determine a candidate font that matches the background image in style from multiple candidate fonts.

[0182] In a possible implementation, the text image generating apparatus 1000 further includes: a first determining unit; the first determining unit is configured to:

[0183] Generating text regions for the background image using a text region generation model according to the aspect ratio of the to-be-processed text in the target font and the saliency image of the background image, thereby obtaining a plurality of first text regions;

[0184] determining a target text region from a plurality of first text regions;

[0185] The rendering unit 1030 is used for:

[0186] In the target text area of ​​the background image, the text to be processed is rendered and generated according to the target font to obtain a text image.

[0187] In a possible implementation, the text image generating apparatus 1000 further includes: a saliency image obtaining unit; the saliency image obtaining unit is configured to:

[0188] Performing segmentation prediction on the background image through the segmentation prediction module in the saliency analysis model to obtain a segmented image of the background image;

[0189] The segmented image is refined and mapped by the refinement mapping module in the saliency analysis model to obtain a saliency image.

[0190] In a possible implementation manner, the first determining unit is configured to:

[0191] determining a plurality of pixel saliency values ​​for each first text region according to the saliency image;

[0192] According to the plurality of pixel significance values ​​of each first text region, a preset number of first text regions having pixel significance values ​​greater than the first significance value are deleted from the plurality of first text regions to obtain a plurality of second text regions;

[0193] determining an average saliency value of each second text region according to the saliency image;

[0194] According to the average significance value of each second text region, a second text region having an average significance value less than the second significance value is selected from the plurality of second text regions to obtain a candidate text region;

[0195] Determine a target text region based on the candidate text regions.

[0196] In a possible implementation manner, the first determining unit is configured to:

[0197] determining an average saliency value of each first text region according to the saliency image;

[0198] According to the average significance value of each first text region, selecting a first text region whose average significance value is less than the second significance value from the plurality of first text regions to obtain a plurality of third text regions;

[0199] determining a plurality of pixel saliency values ​​for each third text region according to the saliency image;

[0200] According to the plurality of pixel significance values ​​of each third text region, a preset number of third text regions having pixel significance values ​​greater than the first significance value are deleted from the plurality of third text regions to obtain candidate text regions;

[0201] Determine a target text region based on the candidate text regions.

[0202] In a possible implementation manner, the first determining unit is configured to:

[0203] If the candidate text region includes multiple candidate text regions, performing aesthetic prediction on each candidate text region of the background image by using an aesthetic prediction model to obtain aesthetic prediction data for each candidate text region;

[0204] A target text region is determined from a plurality of candidate text regions according to the aesthetic prediction data of each candidate text region.

[0205] In a possible implementation, the text image generating apparatus 1000 further includes: a second determining unit; the second determining unit is configured to:

[0206] Perform color clustering on the background image to obtain the main color of the background image;

[0207] Determine the background color of the target text area according to the main color of the background image;

[0208] Determine a target color of the text to be processed from the plurality of candidate colors according to a contrast ratio between the background color and each of the plurality of candidate colors;

[0209] The rendering unit 1030 is used for:

[0210] In the target text area of ​​the background image, the text to be processed is rendered and generated according to the target font and the target color to obtain a text image.

[0211] In a possible implementation, the text image generating apparatus 1000 further includes: a keyword and non-keyword acquiring unit; the keyword and non-keyword acquiring unit is used to:

[0212] Obtain the keywords to be processed and the non-keywords to be processed of the text to be processed;

[0213] The second determining unit is configured to:

[0214] According to the contrast between the background color and each candidate color, determining the target color from the plurality of candidate colors includes a first color of the keyword to be processed and a second color of the non-keyword to be processed; the first color is different from the second color;

[0215] The rendering unit 1030 is used for:

[0216] In the target text area of ​​the background image, the keywords to be processed and the non-keywords to be processed are rendered and generated according to the target font, the first color and the second color to obtain a text image.

[0217] In a possible implementation manner, the second determining unit is configured to:

[0218] According to the contrast between the background color and each candidate color, multiple candidate colors are sorted from large to small to obtain a color sorting result;

[0219] Determine the first color according to the first candidate color in the color sorting result;

[0220] The second color is determined according to the second candidate color in the color sorting result.

[0221] In a possible implementation, the keyword and non-keyword obtaining unit is used to:

[0222] Preprocessing the text to be processed by using a second language model to obtain a plurality of to-be-processed word segments and to-be-processed keywords of the text to be processed;

[0223] The keywords to be processed are deleted from the multiple word segmentations to be processed to obtain non-keywords to be processed.

[0224] As can be seen from the above technical solution, the acquisition unit acquires the text to be processed and the background image; the matching unit inputs the visual features of the background image and the attribute features of each candidate font in multiple candidate fonts into the first language model for style matching, and determines the target font that matches the background image in style from the multiple candidate fonts; for the case of rendering the text to be processed in the background image to obtain the text image, considering the visual information of the background image represented by the visual features of the background image, and considering the attribute information of each candidate font represented by the attribute features of each candidate font in multiple candidate fonts, the matching unit can select the candidate font that matches the background image in style from the multiple candidate fonts as the target font through the powerful semantic understanding ability of the first language model. The rendering unit renders the text to be processed in the background image according to the target font to generate a text image, so that the text image includes the background image and the text to be processed in the target font; based on the target font matching the background image in style, the text to be processed is rendered in the background image with the target font to obtain the text image, so that the text to be processed in the target font in the text image and the background image are more harmonious and beautiful, enhancing the visual effect of the text image, thereby improving the generation effect of the text image.

[0225] The present application also provides a computer device, which may be a server. Fig.11 , Fig.11 A structural diagram of a server provided in an embodiment of the present application, the server 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors, such as a central processing unit CPU 1122, and a memory 1132, one or more storage media 1130 (such as one or more mass storage devices) storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 can be short-term storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1122 can be configured to communicate with the storage medium 1130 and execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0226] The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0227] In this embodiment, the central processor 1122 in the server 1100 can execute the methods provided in various optional implementations of the above embodiments.

[0228] The computer device provided in the embodiment of the present application may also be a terminal. Fig.12 , Fig.12 A structural diagram of a terminal provided in an embodiment of the present application. Taking a smartphone as an example, the smartphone includes: a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290 and other components. The input unit 1230 may include a touch panel 1231 and other input devices 1232, the display unit 1240 may include a display panel 1241, and the audio circuit 1260 may include a speaker 1261 and a microphone 1262. Those skilled in the art will understand that Fig.12 The structure of the smartphone shown in the figure does not constitute a limitation of the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0229] The memory 1220 can be used to store software programs and modules. The processor 1280 executes various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, a phone book, etc.), etc. In addition, the memory 1220 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0230] The processor 1280 is the control center of the smartphone, which uses various interfaces and lines to connect various parts of the entire smartphone, and executes various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 1220, and calling data stored in the memory 1220. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1280.

[0231] In this embodiment, the processor 1280 in the smart phone can execute the methods provided in various optional implementations of the above embodiments.

[0232] According to one aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program. When the computer program is executed on a computer device, the computer device executes the methods provided in various optional implementations of the above embodiments.

[0233] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in various optional implementations of the above-mentioned embodiments.

[0234] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0235] The terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0236] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0237] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0238] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0239] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), RAM, disk or optical disk and other media that can store computer programs.

[0240] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0241] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, ordinary technical members in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A text image generation method, characterized in that: The method comprises: Get the text and background image to be processed; Performing style matching on the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts by using a first language model, and determining a target font from the plurality of candidate fonts; the target font matches the background image in style; The text to be processed is rendered in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

2. The method according to claim 1, characterized in that The performing style matching on the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts by using a first language model, and determining a target font from the plurality of candidate fonts, comprises: The first language model performs style matching on the visual features and the attribute features of each candidate font according to style matching hints, and determines the target font from the multiple candidate fonts; the style matching hints are used to guide the first language model to determine a candidate font that stylistically matches the background image from the multiple candidate fonts.

3. The method according to claim 1, characterized in that The method further comprises: Generating text regions for the background image using a text region generation model according to the aspect ratio of the to-be-processed text in the target font and the saliency image of the background image, to obtain the plurality of first text regions; determining a target text region from among the plurality of first text regions; The step of rendering the text to be processed in the background image according to the target font to obtain a text image comprises: In the target text area of ​​the background image, the text to be processed is rendered and generated according to the target font to obtain the text image.

4. The method according to claim 3, characterized in that The step of obtaining the saliency image comprises: Performing segmentation prediction on the background image by using a segmentation prediction module in a saliency analysis model to obtain a segmented image of the background image; The segmented image is refined and mapped by a refinement mapping module in the saliency analysis model to obtain the saliency image.

5. The method according to claim 3, characterized in that: The determining of a target text region from the plurality of first text regions comprises: Determining a plurality of pixel saliency values ​​for each first text region according to the saliency image; According to the plurality of pixel significance values ​​of each first text region, a preset number of first text regions having pixel significance values ​​greater than the first significance value are deleted from the plurality of first text regions to obtain a plurality of second text regions; determining an average saliency value of each second text region according to the saliency image; According to the average significance value of each second text region, selecting a second text region whose average significance value is less than the second significance value from the plurality of second text regions to obtain a candidate text region; The target text region is determined according to the candidate text regions.

6. The method according to claim 4, characterized in that The determining of a target text region from the plurality of first text regions comprises: determining an average saliency value of each first text region according to the saliency image; According to the average significance value of each first text region, selecting a first text region whose average significance value is less than the second significance value from the plurality of first text regions to obtain a plurality of third text regions; determining a plurality of pixel saliency values ​​for each third text region according to the saliency image; According to the plurality of pixel significance values ​​of each third text region, a preset number of third text regions having pixel significance values ​​greater than the first significance value are deleted from the plurality of third text regions to obtain candidate text regions; The target text region is determined according to the candidate text regions.

7. The method according to claim 5 or 6, characterized in that: The step of determining the target text area according to the candidate text area comprises: If the candidate text region includes multiple candidate text regions, performing aesthetic prediction on each candidate text region of the background image by using an aesthetic prediction model to obtain aesthetic prediction data of each candidate text region; A target text region is determined from the plurality of candidate text regions according to the aesthetic prediction data of each candidate text region.

8. The method according to claim 3, characterized in that The method further comprises: Performing color clustering on the background image to obtain the main color of the background image; Determining the background color of the target text area according to the main color of the background image; Determining a target color of the text to be processed from the multiple candidate colors according to a contrast ratio between the background color and each candidate color among the multiple candidate colors; The step of rendering the text to be processed in the target text area of ​​the background image according to the target font to obtain the text image includes: In the target text area of ​​the background image, the text to be processed is rendered and generated according to the target font and the target color to obtain the text image.

9. The method according to claim 8, characterized in that The method further comprises: Obtaining keywords to be processed and non-keywords to be processed of the text to be processed; The step of determining the target color of the text to be processed from the multiple candidate colors according to the contrast between the background color and each candidate color in the multiple candidate colors comprises: According to the contrast between the background color and each of the candidate colors, determining from the multiple candidate colors that the target color includes a first color of the keyword to be processed and a second color of the non-keyword to be processed; the first color is different from the second color; The rendering and generating of the text to be processed in the target text area of ​​the background image according to the target font and the target color to obtain the text image includes: In the target text area of ​​the background image, the keywords to be processed and the non-keywords to be processed are rendered and generated according to the target font, the first color, and the second color to obtain the text image.

10. The method according to claim 9, characterized in that The step of determining, from the plurality of candidate colors, the target color including the first color of the keyword to be processed and the second color of the non-keyword to be processed according to the contrast between the background color and each of the candidate colors comprises: According to the contrast between the background color and each candidate color, the multiple candidate colors are sorted from large to small to obtain a color sorting result; Determine the first color according to the first candidate color in the color sorting result; The second color is determined according to the second candidate color in the color sorting result.

11. The method according to claim 9, characterized in that The step of obtaining keywords to be processed and non-keywords to be processed of the text to be processed includes: Performing text preprocessing on the text to be processed by using a second language model to obtain a plurality of to-be-processed segmented words and the to-be-processed keywords of the text to be processed; The keywords to be processed are deleted from the multiple word segments to be processed to obtain the non-keywords to be processed.

12. A text image generating device, characterized in that: The device comprises: An acquisition unit, used for acquiring the text to be processed and the background image; a matching unit, configured to perform style matching on the visual features of the background image and the attribute features of each candidate font among a plurality of candidate fonts through a first language model, and determine a target font from the plurality of candidate fonts; the target font matches the background image in style; A rendering unit is used to render the text to be processed in the background image according to the target font to obtain a text image; the text image includes the background image and the text to be processed in the target font.

13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1 to 11 according to the instructions in the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed on a computer device, the computer device executes the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed on a computer device, the computer device is caused to execute the method according to any one of claims 1 to 11.

Images