A text image generation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a text image generation method and device, electronic equipment and storage medium, the method comprises: obtaining an image to be processed; obtaining a rule text to be added input by a user, or generating a rule text to be added according to a preset text generation rule, and splitting the rule text to be added into characters as target characters; based on the position of the text line in the image to be processed, extracting a text line image from the image to be processed; according to the pixel value of the characters in the text line image, replacing the characters with the target characters to obtain a processed text line image; and synthesizing the processed text line image and the image to be processed to obtain a target text image. The above method can make the generation of the text image more flexible and achieve the purpose of generating a target text image including a text meeting certain rules, and the synthesized target text image has good consistency with other regions in the text region, and the quality of the generated target text image is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of text image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for generating text images. Background Technology

[0002] Text recognition, as a common automatic recognition technology, can locate and read text in images, thereby realizing the automatic collection and management of information. With the rapid development of high-performance computing hardware, deep learning technology in the field of machine vision has become increasingly popular. Due to its ability to fit features and its strong anti-interference characteristics, CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network) are gradually being used in the field of text recognition.

[0003] However, deep learning networks are complex and have a large number of parameters, requiring substantial training data to achieve strong generalization performance. In practical applications, data collection is not easy. For example, in industrial settings, production data may involve protected commercial information, so the available data is often insufficient to train deep learning networks for text localization and recognition. Even in scenarios where sufficient data can be collected, the rapid and efficient annotation and cleaning of large amounts of data remains a critical issue. Therefore, scene text editing technology has emerged. This technology can generate a new sample image containing text by fusing existing text images with the given text, thereby increasing the number of sample images and meeting the training requirements of deep learning networks.

[0004] However, generating sample images by fusing given text images requires a large number of text images, which limits the flexibility of text image generation. Furthermore, the background texture and other image parameters of the given text images and sample images may differ significantly, which may result in poor image fusion effects, i.e., poor quality of the generated sample images. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for generating text images, so as to improve the flexibility and image quality of text image generation. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a method for generating a text image, the method comprising:

[0007] Obtain the image to be processed;

[0008] Obtain the text of the rule to be added input by the user, or generate the text of the rule to be added according to the preset text generation rules, and divide the text of the rule to be added into characters as target characters;

[0009] Based on the position of the text lines in the image to be processed, extract the text line image from the image to be processed;

[0010] Based on the pixel values ​​of the characters in the text line image, the target character is used to replace the character, resulting in a processed text line image;

[0011] The processed text line image is combined with the image to be processed to obtain the target text image.

[0012] Optionally, the step of replacing the character with the target character based on the pixel value of the character in the text line image to obtain the processed text line image includes:

[0013] Based on the background texture of the text line image, generate a background image corresponding to the text line image;

[0014] The target pixel value of the target character is determined based on the pixel values ​​of the characters in the text line image;

[0015] The target character is added to the background image according to the target pixel value to obtain the processed text line image.

[0016] Optionally, the step of generating a background image corresponding to the text line image based on the background texture of the text line image includes:

[0017] Determine the position information of each character in the text line image;

[0018] A text foreground mask is determined based on the location information, wherein the text foreground mask is used to identify the text foreground and background in the text line image;

[0019] The background texture of the text foreground portion of the text line image that does not belong to the text foreground mask is filled into the text foreground mask to generate the background image corresponding to the text line image.

[0020] Optionally, the step of determining the position information of each character in the text line image includes:

[0021] Based on the image features of the text line image, determine the position information of each character in the text line image; or,

[0022] Obtain the position information of each character in the image of the text line input by the user.

[0023] Optionally, the step of determining the text foreground mask based on the location information includes:

[0024] The location information is fused to obtain the foreground region of the text line;

[0025] The foreground region of the text line is binarized to obtain a text foreground mask.

[0026] Optionally, the step of determining the target pixel value of the target character based on the pixel values ​​of the characters in the text line image includes:

[0027] Extract the pixel values ​​corresponding to the characters from the text line image and use them as the target pixel values ​​of the target characters; or, perform erosion processing on the text foreground mask to obtain a text skeleton image.

[0028] Calculate the average value of the pixels in the region corresponding to the text skeleton region in the text skeleton image in the text line image, and use it as the target pixel value of the target character.

[0029] Optionally, the step of adding the target character to the background image according to the target pixel value to obtain the processed text line image includes:

[0030] The system can either search for the character mask corresponding to each target character in a preset character library, or generate the character mask corresponding to each target character based on a user-specified font. The preset character library stores the character masks corresponding to each character.

[0031] The target pixel value is assigned to the character region identified by the character mask, and the assigned character region is added and blended into the background image to obtain the processed text line image.

[0032] Optionally, after the step of adding the target character to the background image according to the target pixel value, the method further includes:

[0033] Record the position information of the target character added to the background image and the content information of the target character, and use them as the label corresponding to the processed text line image.

[0034] Optionally, the step of extracting the text line image from the image to be processed based on the position of the text line in the image to be processed includes:

[0035] Based on the position of the text line in the image to be processed, the feature information of the minimum bounding rectangle of the text line in the image to be processed is calculated, wherein the feature information includes the corner coordinates of the minimum bounding rectangle and the width and height of the minimum bounding rectangle;

[0036] Based on the feature information of the minimum bounding rectangle, the feature information of the target rectangle is determined, wherein the target rectangle is a rectangle whose width and height are the same as the width and height of the minimum bounding rectangle, and whose direction is consistent with the pixel arrangement direction of the image to be processed;

[0037] The projection transformation matrix is ​​calculated based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle.

[0038] The image to be processed is transformed by projection according to the projection transformation matrix to obtain the transformed image;

[0039] Extract text lines from the transformed image according to the target rectangle;

[0040] The step of combining the processed text line image with the image to be processed to obtain the target text image includes:

[0041] The processed text line image is then mapped back onto the image to be processed using the projection transformation matrix to obtain the target text image.

[0042] Secondly, embodiments of the present invention provide a text image generation apparatus, the apparatus comprising:

[0043] The image acquisition module is used to acquire the image to be processed.

[0044] The target character determination module is used to obtain the text of the rule to be added input by the user, or to generate the text of the rule to be added according to the preset text generation rules, and to divide the text of the rule to be added into characters as target characters;

[0045] The text line image extraction module is used to extract text line images from the image to be processed based on the position of the text lines in the image to be processed;

[0046] The character replacement module is used to replace the characters in the text line image with the target characters based on the pixel values ​​of the characters in the text line image, so as to obtain the processed text line image;

[0047] The image synthesis module is used to synthesize the processed text line image with the image to be processed to obtain the target text image.

[0048] Optionally, the character replacement module includes:

[0049] The background image generation submodule is used to generate a background image corresponding to the text line image based on the background texture of the text line image;

[0050] The target pixel value determination submodule is used to determine the target pixel value of the target character based on the pixel values ​​of the characters in the text line image;

[0051] The character addition submodule is used to add the target character to the background image according to the target pixel value to obtain the processed text line image.

[0052] Optionally, the background image generation submodule includes:

[0053] A character position information determination unit is used to determine the position information of each character in the text line image;

[0054] A text foreground mask determination unit is used to determine a text foreground mask based on the location information, wherein the text foreground mask is used to identify the text foreground and background in the text line image;

[0055] The background texture filling unit is used to fill the text foreground portion of the text line image that does not belong to the text foreground portion identified by the text foreground mask into the text foreground mask, thereby generating the background image corresponding to the text line image.

[0056] Optionally, the character position information determining unit is specifically used to determine the position information of each character in the text line image based on the image features of the text line image; or, to obtain the position information of each character in the text line image input by the user.

[0057] Optionally, the text foreground mask determination unit includes:

[0058] A text line foreground region determination subunit is used to fuse the position information to obtain the text line foreground region;

[0059] The binarization subunit is used to perform binarization processing on the foreground region of the text line to obtain a text foreground mask.

[0060] Optionally, the target pixel value determination submodule is specifically used to extract the pixel value corresponding to the character from the text line image as the target pixel value of the target character; or, to perform erosion processing on the text foreground mask to obtain a text skeleton image; and to calculate the average value of the pixel points in the text line image corresponding to the text skeleton region in the text skeleton image as the target pixel value of the target character.

[0061] Optionally, the character adding submodule includes:

[0062] The character mask determination unit is used to find the character mask corresponding to each target character from a preset character library, or to generate the character mask corresponding to each target character according to the font specified by the user. The preset character library stores the character mask corresponding to each character.

[0063] The character addition unit is used to assign the target pixel value to the character region identified by the character mask, and add the assigned character region to the background image to obtain the processed text line image.

[0064] Optionally, the device further includes:

[0065] The label determination module is used to record the position information of the target character added to the background image and the content information of the target character, as the label corresponding to the processed text line image.

[0066] Optionally, the text line image extraction module includes:

[0067] The minimum bounding rectangle calculation submodule is used to calculate the feature information of the minimum bounding rectangle of the text line in the image to be processed based on the position of the text line in the image to be processed. The feature information includes the corner coordinates of the minimum bounding rectangle and the width and height of the minimum bounding rectangle.

[0068] The target rectangle determination submodule is used to determine the feature information of the target rectangle based on the feature information of the minimum bounding rectangle, wherein the target rectangle is a rectangle whose width and height are the same as the width and height of the minimum bounding rectangle, and whose direction is consistent with the pixel arrangement direction of the image to be processed;

[0069] The projection transformation matrix determination submodule is used to calculate the projection transformation matrix based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle.

[0070] The image projection transformation submodule is used to perform projection transformation on the image to be processed according to the projection transformation matrix to obtain the transformed image;

[0071] The image cropping submodule is used to crop a text line image from the transformed image according to the target rectangle;

[0072] The image synthesis module is specifically used to reverse map the processed text line image onto the image to be processed according to the projection transformation matrix to obtain the target text image.

[0073] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0074] Memory, used to store computer programs;

[0075] When a processor executes a program stored in memory, it implements the steps of the text image generation method described in any of the first aspects above.

[0076] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the text image generation method described in any of the first aspects.

[0077] Beneficial effects of the embodiments of the present invention:

[0078] In the solution provided by this invention, an electronic device can acquire an image to be processed; acquire user-inputted text to be added as a rule, or generate text to be added as a rule according to a preset text generation rule, and segment the text to be added as a rule into characters as target characters; extract text line images from the image to be processed based on the position of the text lines in the image to be processed; replace characters with target characters according to the pixel values ​​of the characters in the text line images to obtain processed text line images; and synthesize the processed text line images with the image to be processed to obtain a target text image. Since the electronic device can replace the original characters in the text line images with target characters, it can perform character-level replacement. Furthermore, the target characters are obtained from user-inputted text to be added as a rule that conforms to certain rules, or from text generation rules that conform to certain rules. Therefore, the text in the image to be processed can be replaced with target characters that conform to certain rules according to actual needs, offering greater flexibility and achieving the goal of generating a target text image that includes text conforming to certain rules. Furthermore, the electronic device can extract the text line image from the image to be processed, replace the characters in the text line image, and then composite the processed text line image back into the original position of the text line image in the image to be processed. During this process, only the text in the text line image changes; the image parameters of the background remain unchanged. Therefore, the text region containing regular text in the composite target text image has excellent conformity with other regions, greatly improving the quality of the generated target text image. Of course, implementing any product or method of this invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0080] Figure 1 A flowchart illustrating a method for generating a text image according to an embodiment of the present invention;

[0081] Figure 2 for Figure 1 A specific flowchart of step S104 in the illustrated embodiment;

[0082] Figure 3 for Figure 2 A specific flowchart of step S201 in the illustrated embodiment;

[0083] Figure 4 for Figure 2 A specific flowchart of step S203 in the illustrated embodiment;

[0084] Figure 5 for Figure 1 A specific flowchart of step S103 in the illustrated embodiment;

[0085] Figure 6 For based on Figure 1 A flowchart illustrating a method for generating text images according to the embodiment shown;

[0086] Figure 7 Based on Figure 1 A specific flowchart of the text image generation method in the illustrated embodiment;

[0087] Figure 8 This is a schematic diagram of the structure of a text image generation device provided in an embodiment of the present invention;

[0088] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0089] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0090] To improve the flexibility and image quality of text image generation, embodiments of the present invention provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating text images. The method for generating text images provided by embodiments of the present invention will be described first.

[0091] The text image generation method provided in this embodiment of the invention can be applied to any electronic device that needs to generate text images, such as a computer, tablet computer, or mobile phone, without specific limitations. For clarity, it will be referred to as an electronic device below.

[0092] like Figure 1 As shown, a method for generating a text image includes:

[0093] S101, Obtain the image to be processed;

[0094] S102, obtain the text of the rule to be added input by the user, or generate the text of the rule to be added according to the preset text generation rules, and divide the text of the rule to be added into characters as target characters;

[0095] S103, Based on the position of the text line in the image to be processed, extract the text line image from the image to be processed;

[0096] S104, Based on the pixel value of the character in the text line image, replace the character with the target character to obtain the processed text line image;

[0097] S105, the processed text line image is combined with the image to be processed to obtain the target text image.

[0098] As can be seen, in the solution provided by the embodiments of the present invention, the electronic device can acquire an image to be processed; acquire user-inputted text to be added as a rule, or generate text to be added as a rule according to a preset text generation rule, and segment the text to be added as a rule into characters as target characters; extract text line images from the image to be processed based on the position of the text lines in the image to be processed; replace characters with target characters according to the pixel values ​​of the characters in the text line images to obtain processed text line images; and synthesize the processed text line images with the image to be processed to obtain a target text image. Since the electronic device can replace the original characters in the text line images with target characters, it can perform character-level replacement. Furthermore, since the target characters are obtained from user-inputted text to be added conforming to certain rules or text to be added conforming to certain rules generated according to text generation rules, the text in the image to be processed can be replaced with target characters conforming to certain rules according to actual needs, offering greater flexibility and achieving the goal of generating a target text image including text conforming to certain rules. Furthermore, the electronic device can extract the text line image from the image to be processed, replace the characters in the text line image, and then synthesize the processed text line image into the original position of the text line image in the image to be processed. In this process, only the text in the text line image changes, while the image parameters of the background where the text is located do not change. Therefore, the text region containing regular text in the synthesized target text image has good fit with other regions, and the quality of the generated target text image is greatly improved.

[0099] When training text processing models such as text recognition models or text localization models, it is necessary to perform text editing processing on images containing text to change the text in the image and generate a new text image. At this time, the electronic device can acquire the image containing text, that is, execute the above step S101 to acquire the image to be processed. The image to be processed is the image containing text. For the text processing model training scenario, the image to be processed can be a sample image used to train the text processing model.

[0100] In step S102 above, the electronic device can acquire the rule text to be added input by the user, or generate the rule text to be added according to a preset text generation rule, and segment the rule text to be added into characters as target characters. The rule text is text whose number of characters, character type, character arrangement, and other information conform to certain text generation rules. The text generation rule can be preset according to the rules of the text included in the desired text image.

[0101] In one implementation, if the rules of the text included in the desired text image are the same as the rules of the text included in the image to be processed, then the electronic device can perform text recognition on the image to be processed to obtain the text recognition result, and then determine that the rule corresponding to the text recognition result is the pre-set text generation rule.

[0102] The text generation rules mentioned above can include rules such as the number of characters in the text, the type of characters in the text, and the arrangement of characters in the text. For example, the text generation rules could be: the number of characters is 7, the type of the characters is Chinese for the first character, English for the second character, and English or numbers for the third to seventh characters, with a one-character spacing between the second and third characters.

[0103] For example, the text generation rule could also be: the number of characters is 14, all characters are numbers, and the characters are arranged such that there is a connector between the 4th and 5th characters, between the 8th and 9th characters, and between the 12th and 13th characters.

[0104] For example, the text generation rule could also be: the number of characters is 11, the type of the characters is: the 1st to 3rd characters are numbers, the 4th character is English, the 5th to 9th characters are numbers, the 10th character is English, the 11th character is numbers, and the characters are arranged such that there is a one-character gap between the 3rd and 4th characters, a one-character gap between the 7th and 8th characters, and a one-character gap between the 9th and 10th characters.

[0105] In one implementation, a user can input text that conforms to a preset text generation rule, i.e., the text to be added as a rule. For example, if the text generation rule is "the number of characters is 14, all characters are numbers, and the characters are arranged such that there is a connector between the 4th and 5th characters, between the 8th and 9th characters, and between the 12th and 13th characters", the user can input the text "7318-8542-8646-52" that conforms to the rule, and the electronic device can then use the text "7318-8542-8646-52" as the text to be added as a rule.

[0106] In another embodiment, the electronic device can generate a text to be added with rules according to a preset text generation rule. For example, the text generation rule is that "the number of characters is 11 characters, the type of the characters is that the types of the 1st character to the 3rd character are digits, the type of the 4th character is English, the types of the 5th character to the 9th character are digits, the type of the 10th character is English, the type of the 11th character is digits, and the arrangement of the characters is that there is a one-character spacing between the 3rd character and the 4th character, there is a one-character spacing between the 7th character and the 8th character, and there is a one-character spacing between the 9th character and the 10th character". The electronic device can obtain the corresponding number of digit characters and English characters from the character library, and arrange the digit characters and English characters according to the arrangement specified in the text rule to obtain the text "452 C135 05 E7", and use the text "452 C135 05 E7" as the text to be added with rules.

[0107] After the electronic device obtains the text to be added with rules, it can split the text to be added with rules into characters as target characters. For example, if the text to be added with rules is "text52A*3-a", the electronic device can split the text to be added with rules into characters "text", "5", "2", "A", "*", "3", "-", "a", and the electronic device can use these 8 characters as target characters.

[0108] After the electronic device obtains the image to be processed, it can extract the text line image from the image to be processed based on the position of the text line in the image to be processed, that is, perform the above step S103. In one embodiment, when the image to be processed is a sample image for training a text processing model, since the sample image has a label, and the label includes the position information of the text in the sample image and the text content, the electronic device can, after obtaining the image to be processed, determine the position of the text line in the image to be processed according to the label of the image to be processed, and then intercept the text line area from the image to be processed according to the position information of the text line to obtain the text line image.

[0109] In another embodiment, when the image to be processed does not have a label, the electronic device can, after obtaining the image to be processed, use a text localization algorithm to localize the text in the image to be processed to determine the position information of the text line in the image to be processed, or send the image to be processed to the user, and the user manually marks the position information of the characters in the text line in the image to be processed. Then, according to the position information of the characters in the text line, the position information of the text line is determined, and further, according to the position information of the text line, the text line area is intercepted from the image to be processed to obtain the text line image.

[0110] After the electronic device extracts the text line image from the image to be processed, it can replace the characters in the text line image with target characters based on the pixel values ​​of the characters in the text line image to obtain the processed text line image, i.e., perform the above step S104. In one embodiment, the pixel values ​​of the target characters can be converted based on the pixel values ​​of the characters in the text line image so that the pixel values ​​of the target characters are consistent with the pixel values ​​of the characters in the text line image. Then, the original characters in the text line image are replaced with the target characters with converted pixel values ​​to obtain the processed text line image. The pixel values ​​can be grayscale values ​​or color values, and are not specifically limited here.

[0111] Of course, depending on actual needs and according to preset rules, the pixel values ​​of the target characters can be converted using the pixel values ​​of the characters in the text line image to make the pixel values ​​of the target characters inconsistent with those of the characters in the text line image. The preset rules can be adding or subtracting a certain value from the pixel values ​​of the characters in the text line image, or performing other types of operations; all of these are reasonable and not specifically limited here. For example, if the pixel values ​​of the characters in the text line image are grayscale values, and the grayscale value is 100, then according to the preset rules, when converting the pixel values ​​of the target characters based on the pixel values ​​of the characters in the text line image, the grayscale value of the characters in the text line image can be increased by 10, and the grayscale value of 110 can be used as the pixel value of the target character. This method can not only change the content of the text but also change its pixel values, thereby increasing the types of target text images generated in subsequent steps and improving the diversity of target text images.

[0112] In step S105 above, after the electronic device obtains the processed text line image, it can merge the processed text line image into the position of the text line image in the image to be processed to obtain the target text image. At this time, the target text image is the image obtained after editing the text in the image to be processed.

[0113] In the solution provided by this invention, when the electronic device edits the text in the image to be processed, it replaces the characters in the image with target characters. This allows for character-level replacement, and the target characters are obtained from user-inputted text that conforms to certain rules or text generated according to text generation rules that conforms to certain rules. Therefore, this improves the flexibility of text editing and enables the generation of a target text image that includes text conforming to certain rules. Furthermore, because the electronic device only modifies the text in the text line image extracted from the image to be processed, without changing the image parameters of the background where the text is located, the processed text line image can be well matched with the image to be processed when the text line image after character replacement processing is merged into the image to be processed, thus greatly improving the quality of the generated target text image.

[0114] As one embodiment of the present invention, such as Figure 2 As shown, the step of replacing the character with the target character based on the pixel value of the character in the text line image to obtain the processed text line image may include:

[0115] S201, Based on the background texture of the text line image, generate a background image corresponding to the text line image.

[0116] When an electronic device replaces characters in a text line image, it can first generate a background image for the text line image based on the background texture of the text line image. Subsequently, the target character can be added to the background image. This ensures that the background texture of the text line image after character replacement is consistent with that of the text line image in the image to be processed. This ensures that when the processed text line image is merged into the image to be processed, the processed text line image can fit well with the image to be processed.

[0117] S202, determine the target pixel value of the target character based on the pixel values ​​of the characters in the text line image.

[0118] In one implementation, the electronic device can determine the target pixel value of a target character based on the pixel values ​​of characters in a text line image, the target pixel value being consistent with the pixel values ​​of characters in the text line image.

[0119] In another embodiment, the electronic device can also determine the target pixel value of the target character based on the pixel values ​​of the characters in the text line image according to a preset rule. This target pixel value is not the same as the pixel values ​​of the characters in the text line image. For example, if the pixel values ​​of the characters in the text line image are color values ​​and the R, G, and B component values ​​are 100, 150, and 135 respectively, then when determining the target pixel value of the target character based on the pixel values ​​of the characters in the text line image according to the preset rule, the R, G, and B component values ​​can be increased by 10, 20, and 30 respectively, and the pixel values ​​with R, G, and B component values ​​of 110, 120, and 165 respectively can be used as the target pixel values ​​of the target character.

[0120] S203, the target character is added to the background image according to the target pixel value to obtain the processed text line image.

[0121] After determining the target pixel value of the target character, the electronic device can add the target character to the background image obtained in step S201 according to the target pixel value to obtain the processed text line image. Compared with the original text line image, the background texture and other image parameters of the processed text line image have not changed before and after processing.

[0122] In the solution provided by this invention, the electronic device generates a background image corresponding to the text line image based on the background texture of the text line image; determines the target pixel value of the target character based on the pixel value of the character in the text line image; and adds the target character to the background image according to the target pixel value to obtain the processed text line image. This ensures that during text editing, only the text in the text line image is changed, without altering the image parameters of the background. Consequently, when the processed text line image is combined with the image to be processed, it maintains a high degree of fit, guaranteeing that the edited image has high quality.

[0123] As one embodiment of the present invention, such as Figure 3 As shown, the step of generating a background image corresponding to the text line image based on the background texture of the text line image may include:

[0124] S301, determine the position information of each character in the text line image.

[0125] When determining the position information of each character in a text line image, depending on different usage needs, the electronic device can automatically locate the position information of the characters in the text line image, or the text line image can be sent to the user, who can input the character positioning information through an input device, that is, the user can manually specify the position information of the characters in the text line image. Both of these are reasonable and will not be specifically limited here.

[0126] The final position information of each character can be represented by the set {box1, box2, ..., box...} n} indicates that the box i The box representing the position of the i-th character, i.e., the rectangle. i =(x c ,y c ,box_w,box_h), where, (x c ,y c The commas (,) represent the coordinates of the center point of the rectangle, and box_w and box_h represent the width and height of the rectangle, respectively. The positioning box can also be represented using the coordinates of the four corner points of the rectangle, etc., and is not specifically limited here.

[0127] In one implementation, the electronic device can automatically locate the position information of characters in a text line image by using the following steps:

[0128] Step a: Perform noise reduction processing on the text line image.

[0129] The noise reduction process can be either histogram equalization or contrast enhancement; no specific limitation is made here. Step a can suppress the impact of noise on the image, improve image quality, and thus enhance the accuracy of the electronic device in determining the position information of characters in subsequent processes.

[0130] Step b: Extract image features from the denoised text line image and perform statistical analysis to obtain the character position information, or input the denoised text line image into a pre-trained character localization network, and output the character position information through the character localization network.

[0131] When extracting image features, machine learning algorithms can be used to extract image features from text lines. For example, image binarization features, maximum stable extremum region features, or stroke width variation features can be used as image features.

[0132] The maximum stable extremum region feature can be extracted using the maximum stable extremum region detection method. Specifically, the electronic device can binarize the text line image to obtain a binarized image. The maximum stable extremum region of this binarized image is then detected to obtain the maximum stable extremum region detection result. Based on the maximum stable extremum region detection result, connected component analysis is performed to obtain multiple connected components. The bounding rectangle of each connected component is then determined, and each bounding rectangle represents the position information of the character corresponding to that connected component.

[0133] S302, determine the text foreground mask based on the location information.

[0134] After determining the positional information of each character in the text line image, this positional information can be merged, that is, the bounding boxes of each character can be merged. For example, the distance between each bounding box can be calculated, and bounding boxes with a distance less than a preset threshold can be merged as a whole. After merging the bounding boxes with a distance less than the preset threshold as a whole, several overall bounding boxes that identify the positions of the characters can be obtained. The area identified by these several overall bounding boxes is the foreground area of ​​the text line.

[0135] The electronic device can then binarize the image within the foreground region of the text line to obtain a text foreground mask. The binarization process can employ methods such as a fixed threshold or Otsu's method, without specific limitations. For example, the electronic device can set the pixel values ​​within the foreground region of the text line that are greater than a preset threshold to 255 (white), and set the pixel values ​​within the foreground region that are not greater than the preset threshold to 0 (black). Then, it can also set the pixel values ​​of the portion of the text line image excluding the foreground region to 0 (black). This results in a text foreground mask where only the positions of the characters are white, and the rest are black, thus clearly distinguishing the text foreground and background in the text line image.

[0136] To improve the accuracy of the position information of each character in the determined text line image, the electronic device can filter the position information of each character in the text line image determined in step S301 before fusing the position information.

[0137] In one implementation, the electronic device can calculate a series of shape parameters for each positioning box, such as aspect ratio and area, based on the determined character position information, and filter out some areas with low confidence in the shape parameters.

[0138] Because the position information of a character is represented in the form of a positioning box, and the position information of the character includes the width and height of the positioning box, areas that are obviously not part of the character positioning box can be filtered out by calculating the aspect ratio, area, etc. of each positioning box. These areas include areas with an aspect ratio that is too large or too small, and areas with an area that is too large or too small.

[0139] In another implementation, the electronic device can also statistically sort the center point positions of each positioning box based on the determined character position information, and filter out positioning boxes that are obviously not in the same text line.

[0140] Since characters on the same line of text should be aligned on the same straight line, the center point of each character's bounding box should also be aligned on the same straight line. Therefore, the center points of each bounding box can be statistically sorted, and bounding boxes that are clearly not on the same line of text can be filtered out. For example, in a two-dimensional coordinate system, the center point of each character's bounding box can be represented based on its coordinates. Then, linear regression can be performed to determine the regression line, and the distance between each center point and the regression line can be calculated. The bounding boxes of characters whose distances are greater than a preset threshold are considered to be bounding boxes that are clearly not on the same line of text and are therefore filtered out.

[0141] For images containing tags, electronic devices can also verify the position information of each character using the following steps:

[0142] Step c: Compare the number of filtered positioning boxes with the number of characters contained in the text content of the label to verify whether the automatically obtained character positioning results are accurate.

[0143] Since the text content in the text line image is fixed when the image to be processed has a label containing text content, the number of characters is also fixed. Therefore, the number of fixed positioning boxes, i.e. the number of character position information in the fixed text line image, can be compared with the number of characters contained in the text content of the label to determine whether the character positioning result is complete.

[0144] Step d: If the automatically obtained character positioning result is determined to be incomplete through verification, the process can return to the step of determining the position information of each character in the text line image, that is, return to step S301 to re-determine the position information of each character in the text line image, or send the character positioning result and the text line image to the user for manual correction.

[0145] If the number of character position information is different from the number of characters contained in the text content of the label, that is, the character positioning result is incomplete, it indicates that there may be omissions or incorrect positioning. At this time, the electronic device can return to step S301 to redetermine the position information of each character in the text line image, or send the character positioning result and the text line image to the user, so that the user can manually correct the character positioning result according to the text line image.

[0146] S303, fill the text foreground portion of the text line image that does not belong to the text foreground portion identified by the text foreground mask into the text foreground mask to generate the background image corresponding to the text line image.

[0147] After determining the text foreground mask, the background texture of the text foreground portion that is not identified by the text foreground mask in the text line image can be filled into the text foreground mask. In this way, the entire text line image after filling will present the background texture, and the filled text line image will then become the background image corresponding to the text line image. The filling method can use the Navier-Stokes method, the TELEA method, or a deep learning inpainting network, etc., and no specific limitation is made here.

[0148] In one implementation, after generating a background image corresponding to a text line image, the electronic device can present the background image to the user through a display device. The user can preview the background image generated by the electronic device through the display device and use the interactive module to correct the background image to ensure that the generated background image maintains a high degree of similarity to the original background image of the text line image.

[0149] In the solution provided by the embodiments of the present invention, the electronic device can determine the position information of each character in the text line image; then determine the text foreground mask based on the position information; and finally fill the background texture of the background part into the text foreground mask to generate the background image corresponding to the text line image. In this way, the background image generated is highly similar to the original background image of the text line image, and a good fit can be guaranteed when the background image is subsequently merged into the image to be processed.

[0150] As one embodiment of the present invention, the step of determining the target pixel value of the target character based on the pixel values ​​of the characters in the text line image may include:

[0151] Extract the pixel values ​​corresponding to the characters from the text line image, and use them as the target pixel values ​​of the target character. Alternatively, perform erosion processing on the text foreground mask to obtain a text skeleton image;

[0152] Calculate the average value of the pixels in the region corresponding to the text skeleton region in the text skeleton image in the text line image, and use it as the target pixel value of the target character.

[0153] In one implementation, the electronic device can directly extract a pixel at the location of a character in the text line image and use the pixel value of that pixel as the target pixel value of the target character. Alternatively, it can randomly extract multiple pixels at the location of the character and then calculate the average pixel value of these multiple pixels as the target pixel value of the target character. Both methods are reasonable and not specifically limited here. This approach allows for the rapid determination of the target pixel value of the target character, improving the efficiency of text image generation.

[0154] Considering that the pixel values ​​at the character edges may differ from the actual pixel values ​​of the character, extracting pixel values ​​from the character edges when determining the target pixel value of the target character could lead to a significant deviation between the determined target pixel value and the actual pixel value of the current character. Therefore, in another implementation, the electronic device can first construct an image erosion kernel and apply image erosion technology multiple times to the text foreground mask to remove the edge regions of the characters, retaining only the main body of the characters, thereby obtaining a text skeleton image. Then, based on the position of the text skeleton region in the text skeleton image, the corresponding region in the text line image is determined. This region is the main body region of the character after removing the character edge regions. Finally, the average pixel value of the pixels in this region can be calculated as the target pixel value of the target character.

[0155] This method ensures that the selected pixel values ​​are located within the main body of the character when determining the target pixel value, avoiding a large deviation between the determined target pixel value and the actual pixel value of the current character due to the extraction of pixel values ​​at the character edges, thereby improving the accuracy of pixel value extraction.

[0156] In one implementation, the electronic device can calculate the pixel average of multiple pixels using the following formula:

[0157]

[0158] Where val_f is the calculated average pixel value, C is the total number of pixels, and I t For the text line image, P i For points within the skeleton region of the foreground text mask, I t (P i ) represents point P on the text line image. i The gray value corresponding to the coordinates of I skelekon This represents the skeleton region of the foreground text mask.

[0159] In this embodiment of the invention, the electronic device can choose to directly extract pixel values ​​as the target pixel values ​​of the target character according to different application requirements, so as to improve the efficiency of text image generation. Alternatively, it can choose to erode the text foreground mask to obtain a text skeleton image, and extract pixel values ​​from the text skeleton image as the target pixel values ​​of the target character, thereby improving the accuracy of pixel value extraction and increasing the flexibility of the text image generation method.

[0160] As one embodiment of the present invention, such as Figure 4As shown, the step of adding the target character to the background image according to the target pixel value to obtain the processed text line image may include:

[0161] S401: Find the character mask corresponding to each target character from the preset character library, or generate the character mask corresponding to each target character according to the font specified by the user.

[0162] In one implementation, a preset character library can pre-store character masks corresponding to each character. These character masks are obtained by binarizing the character image. After acquiring a target character, the electronic device can search for the corresponding character mask from the preset character library. For example, it can search for character masks 1, 2, 3, and 4 corresponding to target characters a, b, c, and d, respectively.

[0163] In another implementation, after acquiring the target characters, the electronic device can generate a character mask corresponding to each target character based on the font specified by the user. For example, if the user-specified font is Times New Roman, after acquiring target characters a, b, c, and d, the electronic device can perform binarization processing on the character images corresponding to the target characters a, b, c, and d using the Times New Roman font to obtain character masks 1, 2, 3, and 4 corresponding to the target characters a, b, c, and d. Of course, after generating the character mask corresponding to each target character based on the user-specified font, the electronic device can also store the character masks in a preset character library to update the character masks in the preset character library.

[0164] S402, the target pixel value is assigned to the character region identified by the character mask, and the assigned character region is added to the background image to obtain the processed text line image.

[0165] After determining the character mask corresponding to the target character, the target pixel value of the target character can be assigned to the character region identified by the character mask so that the pixel value of the character region is consistent with the pixel value of the character in the original text line image. Then, the assigned character region is added to the background image and blended to obtain the processed text line image. Thus, character replacement in the text line image is achieved while ensuring that the background texture and character pixel value of the text line image remain unchanged.

[0166] To ensure a smooth transition of the background region when the assigned character region is added to and blended into the background image, the Poisson blending method can be used for blending. Alternatively, after adding and blending the assigned character region to the background image, a smoothing algorithm can be used to smooth the background image with the added and blended character region. Both of these are reasonable and will not be specifically limited here.

[0167] In this embodiment of the invention, the electronic device searches for the character mask corresponding to each target character from a preset character library, assigns the target pixel value to the character region identified by the character mask, and then adds the assigned character region to the background image to obtain the processed text line image. This can achieve character replacement in the text line image without changing the background texture and character pixel values, ensuring that the parameters of the image do not change after text replacement, thereby ensuring the quality of the image after text editing.

[0168] As one embodiment of the present invention, after the step of adding the target character to the background image according to the target pixel value, the following may be included:

[0169] Record the position information of the target character added to the background image and the content information of the target character, and use them as the label corresponding to the processed text line image.

[0170] After adding target characters to a background image according to their target pixel values, resulting in a processed text line image, the electronic device can save the positional information of the target characters in the background image and their content information. For example, the position coordinates of character 'a', character 'b', character 'c', and character 'd' in the background image, along with the content of the target characters, are recorded as follows: abcd. Furthermore, the electronic device can establish a correspondence between this positional and content information and the processed text line image, using this information as a label for the processed text line image.

[0171] In one embodiment, the electronic device can store the processed text line images and their corresponding labels in a sample database. Subsequently, the text processing model can be trained based on the text line images stored in the sample database and the labels corresponding to each text line image, which contain the position information of the characters in the image and the content information of the characters.

[0172] In this embodiment of the invention, after generating the target text image, the electronic device can record the position information of the target character added to the background image and the content information of the target character, which serve as the label corresponding to the processed text line image. In this way, the electronic device can directly use the target text image as a sample image and use its corresponding label to train the text processing model. Furthermore, since the target character can be obtained from the user-inputted text that conforms to certain rules or the text that conforms to certain rules generated according to the text generation rules, it is more flexible and can achieve the purpose of generating target text images that include text that conforms to certain rules, thus providing a large number of training samples for the training of the text processing model.

[0173] As one embodiment of the present invention, such as Figure 5 As shown, the step of extracting text line images from the image to be processed based on the position of the text lines in the image to be processed may include:

[0174] S501, Based on the position of the text line in the image to be processed, calculate the feature information of the minimum bounding rectangle of the text line in the image to be processed.

[0175] Since text lines appear in various positions in an image, usually not horizontally or vertically, it is not possible to directly crop the text line image by obtaining certain row and column information of the image data matrix. Therefore, it is necessary to use image transformation technology to transform the image to be processed first, and then crop the text line image from the transformed image to be processed.

[0176] Therefore, in the process of extracting text lines from an image to be processed, the electronic device can first calculate the feature information of the minimum bounding rectangle of the text lines in the image to be processed based on the position of the text lines. This feature information can include the corner coordinates of the minimum bounding rectangle as well as the width and height of the minimum bounding rectangle. For example, the coordinates of the four corners of the minimum bounding rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), respectively, and the height of the rectangle is h and the width is w.

[0177] S502, Based on the feature information of the minimum bounding rectangle, determine the feature information of the target rectangle.

[0178] After determining the feature information of the minimum bounding rectangle, the electronic device can determine the feature information of the target rectangle based on this feature information. The width and height of the target rectangle are the same as those of the minimum bounding rectangle, and its direction is consistent with the pixel arrangement direction of the image to be processed. That is, the determined target rectangle is the same size as the minimum bounding rectangle, but its direction is different. The direction of the target rectangle in the image is horizontal or vertical. For example, the coordinates of the four corner points of the target rectangle are (x1', y1'), (x2', y2'), (x3', y3'), and (x4', y4'). Under the premise that the size of the target image should be the same as the minimum bounding rectangle and its direction is consistent with the pixel arrangement direction of the image to be processed, the values ​​of the above coordinates can be x1' = 0, y1' = 0, x2' = w, y2' = 0, x3' = w, y3' = h, x4' = 0, y4' = h.

[0179] S503, the projection transformation matrix is ​​calculated based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle.

[0180] Using the correspondence between the coordinates of the four corner points (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the minimum bounding rectangle and the coordinates of the four corner points (x1', y1'), (x2', y2'), (x3', y3'), and (x4', y4') of the target rectangle, the electronic device can calculate the projection transformation matrix between the minimum bounding rectangle and the target rectangle. The method for calculating the projection transformation matrix can be the conventional method in this field, and will not be elaborated here.

[0181] S504, the image to be processed is transformed by projection according to the projection transformation matrix to obtain the transformed image.

[0182] S505, extract the text line image from the transformed image according to the target rectangle.

[0183] Because the above projection transformation matrix is ​​calculated based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle, it represents the transformation relationship between the minimum bounding rectangle and the target rectangle. Therefore, when performing projection transformation on the image to be processed according to this projection transformation matrix, the direction of the text line image in the image to be processed can be transformed into the direction of the target rectangle. Since the minimum bounding rectangle of the text line image is the same size as the target rectangle, the text line image can be directly extracted from the transformed image using the target rectangle.

[0184] Accordingly, the step of combining the processed text line image with the image to be processed to obtain the target text image may include:

[0185] The processed text line image is then mapped back onto the image to be processed using the projection transformation matrix to obtain the target text image.

[0186] Since the text line image is extracted from the transformed image to be processed, before merging the processed text line image with the image to be processed, the processed text line image needs to be transformed in reverse according to the projection transformation matrix to the direction of the original text line image. Then, the transformed processed text line image is merged with the image to be processed to obtain the target text image.

[0187] In this embodiment of the invention, when the text line does not appear horizontally or vertically in the image to be processed, the electronic device can first transform the image to be processed, then crop the text line image, and finally combine the processed text line image with the image to be processed. Alternatively, the processed text line image can be transformed in reverse, and then the transformed image can be combined with the image to be processed. In this way, when the electronic device acquires the image to be processed, it can acquire an image with any text line direction as the image to be processed, which greatly increases the applicability of the solution.

[0188] The following is based on Figure 6 The text image generation method provided in this embodiment of the invention will be described using an example. The text image generation method provided in this embodiment of the invention may include process I, process II, process III, and process IV, namely, image and label data input, character-level automatic or semi-automatic positioning, input of the text to be added as a rule, and output of the target text image.

[0189] In process I, the image 601 containing text and its text label 602 can be input into the computer processing unit 603, i.e., the aforementioned electronic device. Furthermore, in process II, the computer processing unit 603 can perform automated character positioning, or the computer interactive device 604 can display the image 601 to be processed to the user and then receive the position information of the characters manually positioned by the user, achieving semi-automatic character positioning. Thus, the computer interactive device 604 can acquire the image 605 containing the positioning frame of the characters.

[0190] In process III, the computer interactive device 604 can determine the rule text 606 to be added, which can be text input by the user that conforms to certain rules, or text generated by the computer processing unit 603 according to the text generation rules that conforms to certain rules. The computer processing unit 603 can replace the characters in the image to be processed 601 with the target characters in the rule text 606 to be added, that is, replace the text in the image to be processed 601 with the rule text 606 to be added. Then, in process IV, the electronic device can output the target text image 607 and its text label 608 obtained by replacing the text in the image to be processed 601 with the rule text 606 to be added.

[0191] In this embodiment of the invention, an automatic or interactive semi-automatic method can be selected according to different scenarios to generate a large number of text-containing images and their labels using the proposed text image generation method, providing a large amount of training data for deep learning-based text localization and recognition algorithms. Furthermore, because the text editing method involves replacing the original characters in the text line image with target characters—which are obtained from user-inputted text conforming to certain rules or text generated according to certain rules—the text in the image to be processed can be replaced with target characters conforming to certain rules as needed, offering greater flexibility and achieving the goal of generating target text images containing text conforming to certain rules. Moreover, in this process, only the text changes; the image parameters of the background remain unchanged. Therefore, the text region in the synthesized target text image blends well with other regions, allowing the new text to naturally integrate into the original text background, resulting in a high-quality image after text editing.

[0192] based on Figure 6 The flowchart illustrating the text image generation method is shown below. Figure 7 The text image generation method provided in this embodiment of the invention will be described in detail using examples.

[0193] The electronic device can acquire the image to be processed 701, locate the text line image 702 based on the label of the image to be processed 701, locate the characters in the text line image 702 in an automatic or semi-automatic manner, determine the character location result, i.e. the position information of the character, filter the location result to obtain the filtered position information, and then display the corresponding rectangle in the text line image based on the filtered position information to obtain image 703.

[0194] The text region, i.e., the foreground region of the text line, is obtained by taking the union of the rectangles in image 703. This yields image 704 containing the foreground region of the text line. The foreground region of the text line in image 704 is binarized, and the pixel values ​​of the parts of the text line image 702 other than the foreground region are also set to 0, i.e., set to black, resulting in a text foreground mask 705. The text foreground mask 705 is then eroded to obtain a text skeleton image 706. The background texture of the text foreground parts in text line image 702 that are not identified by the text foreground mask is then filled into the text foreground mask 705, generating the background image 707 corresponding to the text line image.

[0195] The character mask 708 corresponding to the character in the text to be added is found in the preset character library. The pixel value of the character in the foreground area of ​​the text line is assigned to the character area identified by the character mask 708 to obtain the assigned character area 709. The assigned character area 709 is added and blended into the background image 707 to obtain image 710. All characters in the text to be added are processed in the above way to obtain the edited new text line image 711, that is, the processed text line image. Finally, the new text line image 711 is reverse-mapped onto the image to be processed 701 to obtain the target text image 712.

[0196] In this embodiment of the invention, the electronic device can select automatic or interactive semi-automatic modes according to different scenarios to generate a large number of text-containing images and their labels using the proposed text image generation method, providing a large amount of training data for deep learning-based text localization and recognition algorithms. Furthermore, because the text editing method involves replacing the original characters in the text line image with target characters—which are obtained from user-inputted text conforming to certain rules or text generated according to text generation rules—the text in the image to be processed can be replaced with target characters conforming to certain rules as needed, offering greater flexibility and achieving the goal of generating target text images containing text conforming to certain rules. Moreover, in this process, only the text changes; the image parameters of the background remain unchanged. Therefore, the text region in the synthesized target text image has good compatibility with other regions, allowing the new text to naturally blend into the background of the original text, resulting in a high-quality image after text editing.

[0197] Corresponding to the above-described method for generating text images, this invention also provides a text image generation apparatus. The following describes the text image generation apparatus provided by this invention.

[0198] like Figure 8 As shown, a text image generation apparatus may include:

[0199] Image acquisition module 810 is used to acquire the image to be processed;

[0200] The target character determination module 820 is used to obtain the text of the rule to be added input by the user, or to generate the text of the rule to be added according to the preset text generation rules, and to divide the text of the rule to be added into characters as target characters;

[0201] The text line image extraction module 830 is used to extract text line images from the image to be processed based on the position of the text lines in the image to be processed;

[0202] The character replacement module 840 is used to replace the character with the target character according to the pixel value of the character in the text line image, so as to obtain the processed text line image;

[0203] The image synthesis module 850 is used to synthesize the processed text line image with the image to be processed to obtain the target text image.

[0204] As can be seen, in the solution provided by the embodiments of the present invention, the electronic device can acquire an image to be processed; acquire user-inputted text to be added as a rule, or generate text to be added as a rule according to a preset text generation rule, and segment the text to be added as a rule into characters as target characters; extract text line images from the image to be processed based on the position of the text lines in the image to be processed; replace characters with target characters according to the pixel values ​​of the characters in the text line images to obtain processed text line images; and synthesize the processed text line images with the image to be processed to obtain a target text image. Since the electronic device can replace the original characters in the text line images with target characters, it can perform character-level replacement. Furthermore, since the target characters are obtained from user-inputted text to be added conforming to certain rules or text to be added conforming to certain rules generated according to text generation rules, the text in the image to be processed can be replaced with target characters conforming to certain rules according to actual needs, offering greater flexibility and achieving the goal of generating a target text image including text conforming to certain rules. Furthermore, the electronic device can extract the text line image from the image to be processed, replace the characters in the text line image, and then synthesize the processed text line image into the original position of the text line image in the image to be processed. In this process, only the text in the text line image changes, while the image parameters of the background where the text is located do not change. Therefore, the text region containing regular text in the synthesized target text image has good fit with other regions, and the quality of the generated target text image is greatly improved.

[0205] As one embodiment of the present invention, the character replacement module 840 described above may include:

[0206] The background image generation submodule is used to generate a background image corresponding to the text line image based on the background texture of the text line image;

[0207] The target pixel value determination submodule is used to determine the target pixel value of the target character based on the pixel values ​​of the characters in the text line image;

[0208] The character addition submodule is used to add the target character to the background image according to the target pixel value to obtain the processed text line image.

[0209] As one embodiment of the present invention, the background image generation submodule described above may include:

[0210] A character position information determination unit is used to determine the position information of each character in the text line image;

[0211] A text foreground mask determination unit is used to determine a text foreground mask based on the location information, wherein the text foreground mask is used to identify the text foreground and background in the text line image;

[0212] The background texture filling unit is used to fill the text foreground portion of the text line image that does not belong to the text foreground portion identified by the text foreground mask into the text foreground mask, thereby generating the background image corresponding to the text line image.

[0213] As one embodiment of the present invention, the character position information determination unit described above can be specifically used to determine the position information of each character in the text line image based on the image features of the text line image; or to obtain the position information of each character in the text line image input by the user.

[0214] As one embodiment of the present invention, the above-mentioned text foreground mask determination unit may include:

[0215] A text line foreground region determination subunit is used to fuse the position information to obtain the text line foreground region;

[0216] The binarization subunit is used to perform binarization processing on the foreground region of the text line to obtain a text foreground mask.

[0217] As one embodiment of the present invention, the above-mentioned target pixel value determination submodule can be specifically used to extract the pixel value corresponding to the character from the text line image as the target pixel value of the target character; or, to perform erosion processing on the text foreground mask to obtain a text skeleton image; and to calculate the average value of the pixel points in the text line image corresponding to the text skeleton region in the text skeleton image as the target pixel value of the target character.

[0218] As one embodiment of the present invention, the above-mentioned character addition submodule may include:

[0219] The character mask determination unit is used to find the character mask corresponding to each target character from a preset character library, or to generate the character mask corresponding to each target character according to the font specified by the user. The preset character library stores the character mask corresponding to each character.

[0220] The character addition unit is used to assign the target pixel value to the character region identified by the character mask, and add the assigned character region to the background image to obtain the processed text line image.

[0221] As one embodiment of the present invention, the above-mentioned apparatus may further include:

[0222] The label determination module is used to record the position information of the target character added to the background image and the content information of the target character, as the label corresponding to the processed text line image.

[0223] As one embodiment of the present invention, the text line image extraction module 830 described above may include:

[0224] The minimum bounding rectangle calculation submodule is used to calculate the feature information of the minimum bounding rectangle of the text line in the image to be processed based on the position of the text line in the image to be processed. The feature information includes the corner coordinates of the minimum bounding rectangle and the width and height of the minimum bounding rectangle.

[0225] The target rectangle determination submodule is used to determine the feature information of the target rectangle based on the feature information of the minimum bounding rectangle, wherein the target rectangle is a rectangle whose width and height are the same as the width and height of the minimum bounding rectangle, and whose direction is consistent with the pixel arrangement direction of the image to be processed;

[0226] The projection transformation matrix determination submodule is used to calculate the projection transformation matrix based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle.

[0227] The image projection transformation submodule is used to perform projection transformation on the image to be processed according to the projection transformation matrix to obtain the transformed image;

[0228] The image cropping submodule is used to crop a text line image from the transformed image according to the target rectangle;

[0229] The image synthesis module 850 described above can be specifically used to reverse map the processed text line image onto the image to be processed according to the projection transformation matrix to obtain the target text image.

[0230] This invention also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904. The processor 901, communication interface 902, and memory 903 communicate with each other via the communication bus 904.

[0231] Memory 903 is used to store computer programs;

[0232] The processor 901, when executing the program stored in the memory 903, implements the text image generation method steps described in any of the above embodiments.

[0233] As can be seen, in the solution provided by the embodiments of the present invention, the electronic device can acquire an image to be processed; acquire the text to be added by the user input, or generate the text to be added according to a preset text generation rule, and segment the text to be added into characters as target characters; extract the text line image from the image to be processed based on the position of the text line in the image to be processed; replace the characters with the target characters according to the pixel values ​​of the characters in the text line image to obtain the processed text line image; and synthesize the processed text line image with the image to be processed to obtain the target text image. Since the electronic device can replace the original characters in the text line image with the target characters, it can perform character-level replacement, and the target characters are obtained from the text to be added that conforms to certain rules input by the user or the text to be added that conforms to certain rules generated according to the text generation rule, the text in the image to be processed can be replaced with the target characters that conform to certain rules according to actual needs, which is more flexible and can achieve the purpose of generating a target text image that includes text that conforms to certain rules. Furthermore, the electronic device can extract the text line image from the image to be processed, replace the characters in the text line image, and then synthesize the processed text line image into the original position of the text line image in the image to be processed. In this process, only the text in the text line image changes, while the image parameters of the background where the text is located do not change. Therefore, the text region containing regular text in the synthesized target text image has good fit with other regions, and the quality of the generated target text image is greatly improved.

[0234] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0235] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0236] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0237] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0238] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the text image generation method steps described in any of the above embodiments.

[0239] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the text image generation method steps described in any of the above embodiments.

[0240] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0241] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0242] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0243] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for generating a text image, characterized in that, The method includes: Obtain the image to be processed; Obtain the text of the rule to be added input by the user, or generate the text of the rule to be added according to the preset text generation rules, and divide the text of the rule to be added into characters as target characters; Based on the position of the text lines in the image to be processed, extract the text line image from the image to be processed; Based on the pixel values ​​of the characters in the text line image, the target character is used to replace the character, resulting in a processed text line image; The processed text line image is combined with the image to be processed to obtain the target text image; The step of replacing the character with the target character based on the pixel value of the character in the text line image to obtain the processed text line image includes: Based on the background texture of the text line image, generate a background image corresponding to the text line image; The target pixel value of the target character is determined based on the pixel values ​​of the characters in the text line image; The target character is added to the background image according to the target pixel value to obtain the processed text line image; The method further includes, after the step of adding the target character to the background image according to the target pixel value: Record the position information of the target character added to the background image and the content information of the target character, and use them as the label corresponding to the processed text line image; The target text image and its corresponding labels, which contain information about the position of characters in the image and the content of the characters, are used to train the text processing model.

2. The method according to claim 1, characterized in that, The step of generating a background image corresponding to the text line image based on the background texture of the text line image includes: Determine the position information of each character in the text line image; A text foreground mask is determined based on the location information, wherein the text foreground mask is used to identify the text foreground and background in the text line image; The background texture of the text foreground portion of the text line image that does not belong to the text foreground mask is filled into the text foreground mask to generate the background image corresponding to the text line image.

3. The method according to claim 2, characterized in that, The step of determining the position information of each character in the text line image includes: Based on the image features of the text line image, determine the position information of each character in the text line image; or, Obtain the position information of each character in the image of the text line input by the user.

4. The method according to claim 2, characterized in that, The step of determining the text foreground mask based on the location information includes: The location information is fused to obtain the foreground region of the text line; The foreground region of the text line is binarized to obtain a text foreground mask.

5. The method according to claim 2, characterized in that, The step of determining the target pixel value of the target character based on the pixel values ​​of the characters in the text line image includes: Extract the pixel values ​​corresponding to the characters from the text line image, and use them as the target pixel values ​​of the target character; or, The text foreground mask is eroded to obtain a text skeleton image; Calculate the average value of the pixels in the region corresponding to the text skeleton region in the text skeleton image in the text line image, and use it as the target pixel value of the target character.

6. The method according to claim 1, characterized in that, The step of adding the target character to the background image according to the target pixel value to obtain the processed text line image includes: The system can either search for the character mask corresponding to each target character in a preset character library, or generate the character mask corresponding to each target character based on a user-specified font. The preset character library stores the character masks corresponding to each character. The target pixel value is assigned to the character region identified by the character mask, and the assigned character region is added and blended into the background image to obtain the processed text line image.

7. The method according to any one of claims 1-6, characterized in that, The step of extracting text line images from the image to be processed based on the position of the text lines in the image to be processed includes: Based on the position of the text line in the image to be processed, the feature information of the minimum bounding rectangle of the text line in the image to be processed is calculated, wherein the feature information includes the corner coordinates of the minimum bounding rectangle and the width and height of the minimum bounding rectangle; Based on the feature information of the minimum bounding rectangle, the feature information of the target rectangle is determined, wherein the target rectangle is a rectangle whose width and height are the same as the width and height of the minimum bounding rectangle, and whose direction is consistent with the pixel arrangement direction of the image to be processed; The projection transformation matrix is ​​calculated based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle. The image to be processed is transformed by projection according to the projection transformation matrix to obtain the transformed image; Extract text lines from the transformed image according to the target rectangle; The step of combining the processed text line image with the image to be processed to obtain the target text image includes: The processed text line image is then mapped back onto the image to be processed using the projection transformation matrix to obtain the target text image.

8. A text image generation apparatus, characterized in that, The device includes: The image acquisition module is used to acquire the image to be processed. The target character determination module is used to obtain the text of the rule to be added input by the user, or to generate the text of the rule to be added according to the preset text generation rules, and to divide the text of the rule to be added into characters as target characters; The text line image extraction module is used to extract text line images from the image to be processed based on the position of the text lines in the image to be processed; The character replacement module is used to replace the characters in the text line image with the target characters based on the pixel values ​​of the characters in the text line image, so as to obtain the processed text line image; An image synthesis module is used to synthesize the processed text line image with the image to be processed to obtain a target text image; The character replacement module includes: The background image generation submodule is used to generate a background image corresponding to the text line image based on the background texture of the text line image; The target pixel value determination submodule is used to determine the target pixel value of the target character based on the pixel values ​​of the characters in the text line image; The character addition submodule is used to add the target character to the background image according to the target pixel value to obtain the processed text line image; The label determination module is used to record the position information of the target character added to the background image and the content information of the target character, as the label corresponding to the processed text line image; The target text image and its corresponding labels, which contain information about the position of characters in the image and the content of the characters, are used to train the text processing model.

9. The apparatus according to claim 8, characterized in that, The background image generation submodule includes: A character position information determination unit is used to determine the position information of each character in the text line image; A text foreground mask determination unit is used to determine a text foreground mask based on the location information, wherein the text foreground mask is used to identify the text foreground and background in the text line image; The background texture filling unit is used to fill the text foreground portion of the text line image that does not belong to the text foreground portion identified by the text foreground mask into the text foreground mask, thereby generating a background image corresponding to the text line image; and / or, The character position information determination unit is specifically used to determine the position information of each character in the text line image based on the image features of the text line image; or, to obtain the position information of each character in the text line image input by the user; and / or, The text foreground mask determination unit includes: A text line foreground region determination subunit is used to fuse the position information to obtain the text line foreground region; A binarization subunit is used to binarize the foreground region of the text line to obtain a text foreground mask; and / or, The target pixel value determination submodule is specifically used to extract the pixel values ​​corresponding to characters from the text line image, as the target pixel value of the target character; or, to perform erosion processing on the text foreground mask to obtain a text skeleton image; calculate the average value of the pixel points in the text line image corresponding to the text skeleton region in the text skeleton image, as the target pixel value of the target character; and / or The character addition submodule includes: The character mask determination unit is used to find the character mask corresponding to each target character from a preset character library, or to generate the character mask corresponding to each target character according to the font specified by the user. The preset character library stores the character mask corresponding to each character. A character addition unit is used to assign the target pixel value to the character region identified by the character mask, and add the assigned character region to the background image to obtain the processed text line image; and / or, The text line image extraction module includes: The minimum bounding rectangle calculation submodule is used to calculate the feature information of the minimum bounding rectangle of the text line in the image to be processed based on the position of the text line in the image to be processed. The feature information includes the corner coordinates of the minimum bounding rectangle and the width and height of the minimum bounding rectangle. The target rectangle determination submodule is used to determine the feature information of the target rectangle based on the feature information of the minimum bounding rectangle, wherein the target rectangle is a rectangle whose width and height are the same as the width and height of the minimum bounding rectangle, and whose direction is consistent with the pixel arrangement direction of the image to be processed; The projection transformation matrix determination submodule is used to calculate the projection transformation matrix based on the relationship between the corner coordinates of the minimum bounding rectangle and the corner coordinates of the target rectangle. The image projection transformation submodule is used to perform projection transformation on the image to be processed according to the projection transformation matrix to obtain the transformed image; The image cropping submodule is used to crop a text line image from the transformed image according to the target rectangle; The image synthesis module is specifically used to reverse map the processed text line image onto the image to be processed according to the projection transformation matrix to obtain the target text image.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images