Method for embedding characters into text graph

Abstract

The invention relates to the technical field of large-model text maps, in particular to a text embedding method for a text map, which comprises the following steps of: S1, extracting a to-be-embedded text; s2, extracting characters used for generating pictures; the method comprises the following steps: (1) removing embedded characters and feature words; (2) generating a basic picture according to the characters used for generating the picture by applying a text-to-picture large model so as to realize graphical presentation of character description; (3) calculating a proper font size and a proper font style; s3, according to the calculated font size and font, the to-be-embedded characters are drawn on the basic picture, and fusion of the characters and the picture is achieved through an image analysis algorithm. According to the method for combining the large model and the expansion capacity into one application, the customizable picture with clear characters can be directly output, and the image fusion efficiency is improved. The problem that a current large model generates wrong characters is solved.

Description

Technical Field

[0001] The invention relates to the technical field of large-model Vincent graphs, and in particular to a method for embedding text into a Vincent graph. Background Art

[0002] Wenshengtu was originally designed to draw images rather than write or generate text. Therefore, during the training process, no in-depth learning was conducted specifically on the complexity and precision of language and text. In addition, there was a lack of sufficient "labels" for text samples during the training process, which led to insufficient basic data. In addition, the generation of text involves an in-depth understanding of natural language, which is an extremely complex issue. AI may be confused when processing complex language instructions, especially when faced with idioms, proper nouns, or expressions whose literal meaning does not match the actual meaning, which can easily lead to errors. Therefore, it is very difficult to generate text in Wenshengtu. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the present invention provides a method for embedding text into a text image, so as to solve the problem of text errors generated by the current large model.

[0004] To solve the above problems, the technical solution of the present invention is as follows: A method for embedding text into a text image comprises the following steps:

[0005] S1, extracting text to be embedded;

[0006] S2, text extraction for generating images;

[0007] (7) Remove embedded text and feature words;

[0008] (8) By applying the text-generated image model, a basic image is generated according to the text used to generate the image, thereby realizing the graphical presentation of the text description;

[0009] (9) Calculate the appropriate font size and font style;

[0010] S3. Draw the text to be embedded onto the basic image according to the calculated font size and font, and realize the fusion of text and image through image analysis algorithm.

[0011] Furthermore, the step S1 is an extraction step: requirements are made for the input content, the text to be embedded in the image is marked with double quotes, and the text between the double quotes can be extracted using a regular expression during extraction.

[0012] Furthermore, the step S1 is an extraction step: firstly, the common descriptions of adding text to pictures are collected and sorted to construct a vocabulary, and then the input text is segmented, the obtained words or phrases are matched with the vocabulary, the words with corresponding features are identified and the text corresponding to the feature words is extracted.

[0013] Furthermore, the word segmentation process in step S1 adopts the jieba open source solution.

[0014] Furthermore, in step S2, the large model of the Wensheng graph in step (2) is processed using a diffusion model.

[0015] Furthermore, in step (2), the diffusion model processing includes the following steps:

[0016] ①Forward diffusion process;

[0017] ②Reverse diffusion process.

[0018] Furthermore, the step ① includes that in the forward process, the Vincent graph model gradually adds noise to the data until the signal is completely destroyed into noise, wherein each step adds a certain amount of Gaussian noise to the data.

[0019] The mathematical formula is:

[0020]

[0021] Among them, x0 is the original data, x t is the data state at time step t, β t is a small predefined constant that controls the amount of noise added, I is the identity matrix, and N represents a Gaussian distribution.

[0022] Furthermore, the step ② includes, in the reverse process, the model learns how to recover the original data from the noise, which is achieved by a parameterized Markov chain that attempts to reverse the noise addition step in the forward process.

[0023] Mathematically, this can be expressed as learning a model p θ , the model predicts that given x t and time t x t-1 Distribution:

[0024] p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),∑θ(x t ,t))

[0025] Among them, μ θ and Σ θ is a function parameterized by a neural network that predicts the mean and variance,

[0026] From x T ~N(0,I) and iteratively apply pθ to gradually denoise until reaching x0, which is the generated image.

[0027] Furthermore, the image analysis algorithm in step S3 adopts an Alpha fusion method.

[0028] Further, the Alpha fusion comprises the following steps:

[0029] a. Read the basic image generated by the Wensheng graph model and obtain the color information of each pair of pixels that need to be fused;

[0030] b. Calculate the Alpha value. If the source image has an Alpha channel, the transparency is calculated based on the Alpha channel value of the corresponding pixel in the source image.

[0031] c. Apply the Alpha fusion formula to calculate the final color of each pixel,

[0032] The formula can be expressed as: Cout = αCsrc + (1-α) Cdst

[0033] Where: Cout is the output color, Csrc is the color of the source image, Cdst is the color of the destination image, and α is the alpha channel value of the source image.

[0034] d. Update the target image and assign the calculated final color value to the corresponding pixel in the target image.

[0035] Compared with the prior art, the present invention has the following beneficial effects: based on the text image result, through the extended image analysis algorithm, the target text is drawn to the corresponding position of the image with appropriate color, font size, font, etc., so as to achieve the effect of embedding the text image into the text; the image after the image and text fusion processing is returned to the user, and the user can directly use the image fused with text to achieve the perfect presentation of the image and text information. This image and text fusion method based on the large model of the text image can bring more efficient and accurate image and text fusion solutions to the image processing and design fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the method of the present invention;

[0037] Figure 2 Generate pictures for the prior art of the first embodiment of the present invention;

[0038] Figure 3 This is an application to generate pictures for the first embodiment of the present invention. DETAILED DESCRIPTION

[0039] like Figure 1 As shown, a method for embedding text into a text image comprises the following steps:

[0040] S1. Extraction of text to be embedded:

[0041] By identifying and extracting the text describing the content, it is divided into two parts: one part is the text "used to generate the picture", and the other part is the text "to be embedded in the picture". This method can effectively handle the relationship between text and pictures, and achieve high efficiency and accuracy in the fusion of text and pictures.

[0042] Method for extracting text to be embedded:

[0043] You can make requirements on the input content. For example, the text to be embedded in the image must be marked with double quotes. When extracting, you can use regular expressions to extract the text between the double quotes.

[0044] There is also an automatic extraction solution. First, collect and organize the common descriptions of adding text to the picture to build a vocabulary (such as "write in the picture", "inscription", "embedded text", etc.), and then perform word segmentation on the input text. The word segmentation can use open source solutions such as jieba, match the obtained words or phrases with the vocabulary, identify the words with corresponding features, and extract the text corresponding to the feature words.

[0045] S2. Text extraction for generating images:

[0046] The embedded text and feature words are removed and used as the text "used to generate the image".

[0047] By applying the large model of text and images, basic images are generated according to the text "used to generate images", and the graphical presentation of text descriptions is achieved. The large model of text and images has powerful generation capabilities and can accurately convert text descriptions into images, providing a basis for subsequent text and image fusion.

[0048] According to the generated basic image size and the relationship between the image size and the font size set based on experience, the appropriate font size is calculated to ensure that the text embedded in the image is of an appropriate size and is coordinated with the overall style of the image.

[0049] The large model of Wenshengtu is usually processed using the diffusion model, and the mathematical formula is as follows:

[0050] 1. Forward Diffusion Process

[0051] In the forward process, the model gradually adds noise to the data until the signal is completely corrupted into noise. This process can be represented as a series of Markov chain transitions, where each step adds a certain amount of Gaussian noise to the data.

[0052] Mathematically, this can be expressed as:

[0053]

[0054] Where x0 is the original data (e.g., image), x t is the data state at time step t, β t is a predefined small constant (or a time-varying function) that controls the amount of noise added, I is the identity matrix, and N represents a Gaussian distribution.

[0055] 2. Reverse diffusion process (or generation process)

[0056] In the backward process, the model learns how to recover the original data from the noise. This is usually done via a parameterized Markov chain that attempts to reverse the noise addition steps in the forward process.

[0057] Mathematically, this can be expressed as learning a model p θ , the model predicts that given x t and time t x t-1 Distribution:

[0058] p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),∑θ(x t ,t))

[0059] Among them, μ θ and Σ θ is a function parameterized by a neural network (such as U-Net) that predicts the mean and variance.

[0060] To generate an image from noise, we start with x T ~N(0,I) (i.e. pure noise) and iteratively apply p θ to gradually denoise until reaching x0, which is the generated image.

[0061] The model analyzes the style features of the basic image and calculates the appropriate font style based on the relationship between the image style and the font set based on experience, ensuring that the font of the embedded text matches the image style and improves the overall visual effect.

[0062] S3. Draw the text to be embedded onto the base image based on the calculated font size and font, and use an image analysis algorithm to achieve fusion of the text and the image. Different fusion methods can be used, such as drawing a semi-transparent background or inverting the color pixel by pixel, to ensure that the text and the image are integrated.

[0063] The image analysis algorithm can adopt Alpha fusion as follows:

[0064] The basic idea of ​​alpha fusion is to determine the color of each pixel in the final image based on the alpha channel value (transparency) of the pixel. The alpha channel value is usually a floating point number between 0 and 1, where 0 means completely transparent and 1 means completely opaque.

[0065] Specific steps

[0066] 1. Read the basic image generated by the Wensheng graph model and obtain the color information of each pair of pixels that need to be fused.

[0067] 2. Calculate the Alpha value. If the source image has an Alpha channel, calculate the transparency based on the Alpha channel value of the corresponding pixel in the source image.

[0068] 3. Apply the Alpha fusion formula to calculate the final color of each pixel.

[0069] The formula can be expressed as:

[0070] Cout = αCsrc + (1-α)Cdst

[0071] Where: Cout is the output color (the color in the final image), Csrc is the color of the source image, Cdst is the color of the destination image, and α is the alpha channel value of the source image.

[0072] 4. Update the target image and assign the calculated final color value to the corresponding pixel in the target image.

[0073] Finally, the image after text and image fusion processing is returned to the user, and the user can directly use this image with text fusion to achieve perfect presentation of text and image information. This image and text fusion method based on the text-image big model can bring more efficient and accurate image and text fusion solutions to the image processing and design fields.

[0074] Embodiment 1

[0075] For example, there is the following text: a small grass in the desert, with the text "tenacious" in the upper right corner of the picture. The mainstream text-generated image model generates the following effect: Figure 2 As shown in the above figure, we can see that the drawing effect of text, especially Chinese characters, on large models is not ideal.

[0076] The pictures generated using this application scheme are as follows Figure 3 As shown, through text analysis, the main text of the painting "a small grass in the desert" is extracted, and the text information "tenacious" and the text position "in the upper right corner of the picture" are identified. Then, the large model is used to generate a picture based on "a small grass in the desert", and then the preset font and font size are used to add text to the resulting picture to complete the fusion of information and picture.

[0077] The above specific implementation methods are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to examples, a person skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for embedding text into a text image, characterized in that: The following steps are involved: S1, extracting text to be embedded; S2, text extraction for generating images; (4) Remove embedded text and feature words; (5) By applying the text-generated image model, a basic image is generated based on the text used to generate the image, thereby realizing the graphical presentation of the text description; (6) Calculate the appropriate font size and font style; S3. Draw the text to be embedded onto the basic image according to the calculated font size and font, and realize the fusion of text and image through image analysis algorithm.

2. The method for embedding text into a text image according to claim 1, characterized in that: The step S1 is an extraction step: requirements are made for the input content, and the text to be embedded in the image is marked with double quotes. When extracting, a regular expression can be used to extract the text between the double quotes.

3. The method for embedding text into a text image according to claim 1, characterized in that: The step S1 is an extraction step: firstly, the common descriptions of adding text to pictures are collected and sorted to build a vocabulary, and then the input text is segmented, the words or phrases obtained are matched with the vocabulary, the words with corresponding features are identified and the text corresponding to the feature words is extracted.

4. The method for embedding text into a text image according to claim 3, characterized in that: The word segmentation process in step S1 adopts the jieba open source solution.

5. The method for embedding text into a text image according to claim 1, characterized in that: In the step S2, the large model of the Wensheng graph in step (2) is processed using a diffusion model.

6. The method for embedding text into a text image according to claim 5, characterized in that: In step (2), the diffusion model processing includes the following steps: ①Forward diffusion process; ②Reverse diffusion process.

7. The method for embedding text into a text image according to claim 6, characterized in that: The step ① includes, in the forward process, the Vincent graph model gradually adds noise to the data until the signal is completely destroyed into noise, wherein each step adds a certain amount of Gaussian noise to the data, The mathematical formula is: Among them, x0 is the original data, x t is the data state at time step t, β t is a small predefined constant that controls the amount of noise added, I is the identity matrix, and N represents a Gaussian distribution.

8. The method for embedding text into a text image according to claim 7, characterized in that: The step ② includes, in the reverse process, the model learns how to recover the original data from the noise, which is achieved through a parameterized Markov chain that attempts to reverse the noise addition step in the forward process. Mathematically, this can be expressed as learning a model p θ , the model predicts that given x t and time t x t-1 Distribution: p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),∑θ(x t ,t)) Among them, μ θ and Σ θ is a function parameterized by a neural network that predicts the mean and variance, From x T ~N(0,I) and iteratively apply p θ to gradually denoise until reaching x0, which is the generated image.

9. The method for embedding text into a text image according to claim 1, characterized in that: The image analysis algorithm in step S3 adopts Alpha fusion.

10. The method for embedding text into a text image according to claim 9, characterized in that: The Alpha fusion includes the following steps: a. Read the basic image generated by the Wensheng graph model and obtain the color information of each pair of pixels that need to be fused; b. Calculate the Alpha value. If the source image has an Alpha channel, the transparency is calculated based on the Alpha channel value of the corresponding pixel in the source image. c. Apply the Alpha fusion formula to calculate the final color of each pixel, The formula can be expressed as: Cout = αCsrc + (1-α) Cdst Where: Cout is the output color, Csrc is the color of the source image, Cdst is the color of the destination image, α is the alpha channel value of the source image, d. Update the target image and assign the calculated final color value to the corresponding pixel in the target image.

Images