Text image synthesis method, device, storage medium and electronic device

Abstract

The present disclosure relates to the field of image processing technology, and specifically to a text image synthesis method, device, storage medium and electronic device. The text image synthesis method comprises: reading a text image, performing text detection on the text image to obtain a text font range and a text height range, and performing content detection on the text image to obtain a prohibited patch area; obtaining the text to be synthesized, and generating a text patch corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range; judging whether the text patch meets a preset synthesis condition based on a preset patch mode and the prohibited patch area; when the text patch meets the synthesis condition, placing the text patch in the text image to obtain a synthesized text image. The text image synthesis method provided by the present disclosure can solve the problem of low similarity between the synthesized content and the real sample in text image synthesis.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to a text image synthesis method, a text image synthesis device, a storage medium and an electronic device. Background Art

[0002] In application scenarios such as finance and insurance, the recognition and structuring of text in various images are involved. In the business system, it is necessary to enter and verify the information in important images. Using OCR technology to identify claims images can save manpower and shorten the time for claims.

[0003] In the images uploaded by actual customers, there are many types of characters. Deep learning-based text recognition requires a huge data set, usually tens of millions of text items. However, it is difficult to obtain large-scale data in real business scenarios. Limited by the content of training samples, the text recognition system has poor generalization ability and low recognition accuracy.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a text image synthesis method, a text image synthesis device, a storage medium and an electronic device, aiming to solve the problem of low similarity between synthesized content and real samples in text image synthesis.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of an embodiment of the present disclosure, a text image synthesis method is provided, including: reading a text image, performing text detection on the text image to obtain a text font range and a text height range, and performing content detection on the text image to obtain a prohibited patch area; obtaining a text to be synthesized, and generating a text patch corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range; judging whether the text patch meets a preset synthesis condition based on a preset patch mode and the prohibited patch area; and placing the text patch in the text image to obtain a synthesized text image when the text patch meets the synthesis condition.

[0008] According to some embodiments of the present disclosure, based on the aforementioned scheme, the text to be synthesized is obtained, and a text patch corresponding to the text to be synthesized is generated according to the text to be synthesized, the text font range and the text height range, including: randomly selecting the text to be synthesized from a corpus list; and randomly selecting a text font from the text font range; and randomly determining a height value from the text height range, and determining the text font size according to the height value; and generating the text patch according to the text to be synthesized, the text font and the text font size.

[0009] According to some embodiments of the present disclosure, based on the aforementioned scheme, the method of judging whether the text patch meets the preset synthesis condition based on the preset patch mode and the prohibited patch area includes: determining the first boundary and the second boundary of the text image according to the preset patch mode, and determining a pre-placement coordinate in the text image; determining the pre-placement area of ​​the text patch according to the pre-placement coordinate and the size information of the text patch; when the pre-placement area does not exceed the first boundary and the second boundary and does not block the prohibited patch area, returning that the synthesis condition is met; when the pre-placement area exceeds the first boundary, returning that the synthesis condition is not met.

[0010] According to some embodiments of the present disclosure, based on the aforementioned scheme, the method also includes: when the pre-placement area does not exceed the first boundary but exceeds the second boundary, updating the pre-placement coordinates in a first manner; or when it does not exceed the first boundary and the second boundary but occludes the prohibited patch area, updating the pre-placement coordinates in a second manner; and judging whether the text patch meets the synthesis condition based on the updated pre-placement coordinates.

[0011] According to some embodiments of the present disclosure, based on the aforementioned scheme, the method further includes: when the pasting mode is from left to right and then from top to bottom, the first boundary is the lower boundary, the second boundary is the right boundary, and the pre-placed coordinates are the coordinates of the upper left corner of the text image.

[0012] According to some embodiments of the present disclosure, based on the aforementioned scheme, when the text patch meets the synthesis conditions, after the text patch is placed in the text image, the method also includes: updating the text image and the prohibited patch area based on the placed text patch; obtaining a new text to be synthesized, and generating a new text patch corresponding to the new text to be synthesized according to the new text to be synthesized, the text font range and the text height range; judging whether the new text patch meets the preset synthesis conditions based on the preset patch mode and the updated prohibited patch area; and when the new text patch meets the synthesis conditions, placing the new text patch in the updated text image.

[0013] According to some embodiments of the present disclosure, based on the aforementioned scheme, the text detection on the text image to obtain the text font range and the text height range includes: performing text detection on the text image to obtain a text circumscribed rectangle; identifying the text font in the text circumscribed rectangle to obtain the text font range; and obtaining the text height range based on the height value of the text circumscribed rectangle.

[0014] According to a second aspect of an embodiment of the present disclosure, a text image synthesis device is provided, comprising: a detection module, used to read a text image, and perform text detection on the text image to obtain a text font range and a text height range, and perform content detection on the text image to obtain a prohibited patch area; a patch module, used to obtain a text to be synthesized, and generate a text patch corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range; a judgment module, used to judge whether the text patch meets a preset synthesis condition based on a preset patch mode and the prohibited patch area; and a synthesis module, used to place the text patch in the text image to obtain a synthesized text image when the text patch meets the synthesis condition.

[0015] According to a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the text image synthesis method in the above embodiment is implemented.

[0016] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, characterized in that it includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the text image synthesis method as in the above-mentioned embodiment.

[0017] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:

[0018] In the technical solutions provided by some embodiments of the present disclosure, firstly, the acquired text image is subjected to text detection to obtain the text font range and the text height range, and the content of the text image is subjected to content detection to obtain the prohibited patch area; then the text to be synthesized is obtained, and a text patch is generated according to the text font range and the text height range of the text detection; finally, it is determined whether the text patch can be synthesized to finally obtain the synthesized text image. Based on the text image synthesis method provided by the present disclosure, on the one hand, the generated text patch is obtained based on the result of text detection, so that the generated text patch has a high similarity with the text in the real text image, thereby maintaining the increased text image data volume and the text content style in the original real text image to be unified, and more authentic; on the other hand, the prohibited patch area is obtained by pre-detecting the text image content, and then these prohibited patch areas can be avoided when synthesizing the text patch, which can exclude the interference of the non-blank background area with content in the text image, and improve the quality of text image synthesis; on the other hand, based on the automatic generation of text patches and the use of text patches for text image synthesis, manual annotation samples are avoided, the cost of manual annotation is reduced, and the text image data set is enriched, which is then used to improve the generalization ability and recognition accuracy of the text recognition model.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 A schematic diagram schematically illustrates a flow chart of a text image synthesis method in an exemplary embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram schematically illustrates a text image in an exemplary embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram schematically illustrates a text image detection result in an exemplary embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram schematically illustrates a synthetic text image in an exemplary embodiment of the present disclosure;

[0025] FIG5( a ) schematically shows a schematic diagram of a text patch before adjustment in an exemplary embodiment of the present disclosure;

[0026] FIG5( b ) schematically shows a schematic diagram of a text patch after adjustment in an exemplary embodiment of the present disclosure;

[0027] FIG6( a ) schematically shows another text patch before adjustment in an exemplary embodiment of the present disclosure;

[0028] FIG6( b ) schematically shows another schematic diagram of an adjusted text patch in an exemplary embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram schematically illustrates a flow chart of a text image synthesis method in an exemplary embodiment of the present disclosure;

[0030] Figure 8 A schematic diagram schematically illustrates a card text synthesis sample in an exemplary embodiment of the present disclosure;

[0031] Fig. 9 A schematic diagram schematically illustrates a synthetic sample of a driving license text in an exemplary embodiment of the present disclosure;

[0032] Fig.10 A schematic diagram schematically illustrates a synthetic sample of a medical bill text in an exemplary embodiment of the present disclosure;

[0033] Fig.11 A schematic diagram schematically shows the composition of a text image synthesis device in an exemplary embodiment of the present disclosure;

[0034] Fig.12 A schematic diagram schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure;

[0035] Fig.13 A schematic diagram schematically shows the structure of a computer system of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0037] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] Application scenarios such as finance and insurance involve the recognition and structuring of text in various images. For example, in the underwriting and claims settlement of the insurance industry, customers will upload multiple images, including cards, medical bills, financial bills, screenshots, and other printed materials. In the business system, it is necessary to enter and verify the information in important images. OCR (Optical Character Recognition) technology plays a vital role in the field of image text recognition, automatically entering card information, medical information, etc., assisting in information extraction, building multi-dimensional structured data for users, and establishing knowledge graphs, etc., to achieve important tasks such as automated claims settlement or underwriting.

[0041] OCR character recognition refers to the process in which an electronic device (such as a scanner or digital camera) examines characters printed on paper and then uses character recognition methods to translate the shapes into computer text; that is, the process of scanning text materials and then analyzing and processing image files to obtain text and layout information.

[0042] In the images uploaded by actual customers, there are a wide variety of characters, including Chinese, English, punctuation marks, etc., especially Chinese characters, of which there are thousands of commonly used characters and tens of thousands of uncommon characters. There are a wide variety of character fonts in various images, and text recognition based on deep learning requires a huge data set to support it, usually requiring tens of millions of text items.

[0043] Therefore, the text recognition system is often limited by the number and richness of training samples, resulting in poor generalization ability and low recognition accuracy in complex scenarios. It is difficult to obtain large-scale data in real business scenarios, so text image synthesis is needed to expand the data set. Current text image synthesis methods include the following methods:

[0044] 1. Manually label real samples, but this method consumes a lot of manpower and material resources;

[0045] 2. The GAN (Generative Adversarial Network) style transfer method is used to synthesize samples. However, for images of different business scenarios or changing the font of the synthesized samples, the GAN model needs to be retrained, and the generalization ability of the background extraction model is not high. The extracted background cannot effectively remove the characters in the real image;

[0046] 3. Using deep learning and segmentation technology to align the text of a specific corpus with the background image axis can naturally integrate the text into the natural scene, but it is not suitable for text image synthesis in printed images.

[0047] Therefore, the present disclosure provides a text image synthesis method, which uses real text image data and business-specific corpus to quickly synthesize text image patch samples with higher similarity to real samples, so as to obtain the final synthesized samples to expand the text data set.

[0048] The implementation details of the technical solution of the embodiment of the present disclosure are elaborated in detail below.

[0049] Figure 1 The following is a schematic diagram showing a flow chart of a text image synthesis method in an exemplary embodiment of the present disclosure. Figure 1 As shown, the text image synthesis method includes steps S1 to S4:

[0050] Step S1, reading a text image, performing text detection on the text image to obtain a text font range and a text height range, and performing content detection on the text image to obtain a prohibited patch area;

[0051] Step S2, obtaining the text to be synthesized, and generating a text tile corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range;

[0052] Step S3, judging whether the text patch meets the preset synthesis condition based on the preset patch mode and the forbidden patch area;

[0053] Step S4, when the text patch meets the synthesis condition, placing the text patch in the text image to obtain a synthesized text image.

[0054] In the technical solutions provided by some embodiments of the present disclosure, firstly, the acquired text image is subjected to text detection to obtain the text font range and the text height range, and the content of the text image is subjected to content detection to obtain the prohibited patch area; then the text to be synthesized is obtained, and a text patch is generated according to the text font range and the text height range of the text detection; finally, it is determined whether the text patch can be synthesized to finally obtain the synthesized text image. Based on the text image synthesis method provided by the present disclosure, on the one hand, the generated text patch is obtained based on the result of text detection, so that the generated text patch has a high similarity with the text in the real text image, thereby maintaining the increased text image data volume and the text content style in the original real text image to be unified, and more authentic; on the other hand, the prohibited patch area is obtained by pre-detecting the text image content, and then these prohibited patch areas can be avoided when synthesizing the text patch, which can exclude the interference of the non-blank background area with content in the text image, and improve the quality of text image synthesis; on the other hand, based on the automatic generation of text patches and the use of text patches for text image synthesis, manual annotation samples are avoided, the cost of manual annotation is reduced, and the text image data set is enriched, which is then used to improve the generalization ability and recognition accuracy of the text recognition model.

[0055] The following will describe in more detail the steps of the text image synthesis method in this example implementation with reference to the accompanying drawings and embodiments.

[0056] In step S1, a text image is read, and text detection is performed on the text image to obtain a text font range and a text height range, and content detection is performed on the text image to obtain a prohibited patch area.

[0057] The text image is a real image. The text image includes text content, or a combination of text content and other content. For example, a bill image contains text content; a card image also includes text and other content such as a personal photo.

[0058] A plurality of images may be collected in advance to generate an image list or an image database, and then an image may be traversed through the image list or database or randomly selected as a text image.

[0059] Figure 2 The following is a schematic diagram schematically showing a text image in an exemplary embodiment of the present disclosure. Figure 2 As shown, the obtained text image is a screenshot of an ID card, which includes text information such as name, gender, date of birth, address, ID card number, and also includes avatar information.

[0060] After acquiring the text image, text detection is performed on the text image. The text font range and text height range in the text image can be detected by optimizing a deep learning method, such as a network such as CTPN based on detection box regression, or a network such as PSENet, PANNet, DBNet based on image segmentation.

[0061] The text font range is represented as F_range, which is the collection of font styles of all texts in the text image. For example, the text font range of a text image is F_range=[Bold, Song].

[0062] The text height range is expressed as H_range = [h min ,h max ], that is, the value range of the height value of the circumscribed rectangle of all text bars in the text image, h min h is the minimum height of the bounding rectangle of the text bar in the text image. max The maximum height of the bounding rectangle of the text bar.

[0063] In addition, it is necessary to perform content detection on the text image to obtain the prohibited patch area.

[0064] The forbidden patch area refers to the area in the text image that cannot be used for text patch placement. In order to ensure that all text content is clearly visible when synthesizing text images, sample patch synthesis is not allowed in places where there are text, images, etc. Therefore, it is also necessary to perform content detection on the text image to determine the forbidden patch area.

[0065] The forbidden patch area includes the area where all texts in the text image are located, and when the text image includes other images and other contents, these contents also need to be divided into forbidden patch areas. In addition, other forbidden patch areas can be set according to needs on this basis. For example, when synthesizing text patches, they cannot be close to the border of the text image, and a certain distance of page margins needs to be reserved. Therefore, the area of ​​the page margin can also be divided into forbidden patch areas according to needs.

[0066] In other embodiments of the present disclosure, the prohibited patch area may also include other areas that have been detected by content. For example, when synthesizing a text patch, it cannot be close to the border of the text image, and a certain distance of page margin needs to be reserved. Therefore, the area of ​​the page margin can also be divided into a prohibited patch area according to needs.

[0067] In addition, after detecting the text image and obtaining the forbidden patch area, a verification step can be added. For example, during content recognition, non-important content such as watermarks and shading of the text image may be identified as forbidden patch areas, but watermarks or shading do not actually affect the text patch. Therefore, the detected forbidden patch area can be modified in this verification step to obtain a more accurate forbidden patch area for subsequent text image synthesis.

[0068] Figure 3 The following is a schematic diagram schematically showing a text image detection result in an exemplary embodiment of the present disclosure. Figure 3 As shown in FIG. 1 , the detection result of the text image includes multiple text bounding rectangles corresponding to the text content and an image area, which are all prohibited patch areas.

[0069] It should be noted that when the prohibited patch area is obtained by content detection, the prohibited patch area includes the text area. Since the bounding rectangle of each text bar has been obtained based on the result of text detection, the location of the bounding rectangle of the text bar can be regarded as the text area, and then the prohibited patch area can be divided. Therefore, when the content detection needs to use the result of text detection, text detection can be performed first, and then content detection can be performed.

[0070] Of course, content detection can also be performed using other methods, such as using a pre-trained content detection model for detection, where the input is a text image and the output is a prohibited patch area. In this case, the text detection result is not required. In this case, there is no limit on the execution order of text detection and content detection of text images. The two detections can be performed in sequence or simultaneously.

[0071] In addition, the text detection and content detection may be combined into one detection, that is, the text font range, text height range and prohibited tile range of the text image may be obtained through one detection.

[0072] In step S2, the text to be synthesized is obtained, and a text tile corresponding to the text to be synthesized is generated according to the text to be synthesized, the text font range and the text height range.

[0073] In one embodiment of the present disclosure, the text to be synthesized can be randomly selected from the corpus list. The text to be synthesized includes only text content information, which may include Chinese, English, numbers, punctuation marks, and the like.

[0074] For example, the corpus list includes content information such as "Zhuge Kongming", "Jinkou, Shinan District, Qingdao City, Shandong Province", and "110108188801014410", but does not include information such as the style and size of the text.

[0075] After obtaining the text to be synthesized, a text patch corresponding to the text to be synthesized is generated. The content of the text in the text patch is the content of the text to be synthesized. The font of the text is determined according to the text font range detected by the text image, and the font size of the text is determined according to the text height range detected by the text image, so as to obtain the specific content of the text patch.

[0076] At the same time, after the text content, text font and text font size are determined, the length and width of the circumscribed rectangle of the text strip corresponding to the text patch can be calculated as the size information of the text patch.

[0077] Step S3, judging whether the text patch meets the preset synthesis condition based on the preset patch mode and the forbidden patch area;

[0078] Specifically, after a text patch corresponding to the text to be synthesized is generated, it is determined whether the patch can be synthesized with the text image according to a preset patch mode and a prohibited patch area.

[0079] The preset pasting method is the order in which the text pasting needs to be determined in advance. Generally speaking, if the text image is rectangular, the preset pasting method can be to synthesize the text patches in the order from left to right and then from top to bottom; of course, the pasting method can also be from top to bottom and then from left to right, or from right to left and then from top to bottom, etc.; or when the text image is circular, from the center to the outside, etc.

[0080] Different preset patch modes have different bases for judging whether a text patch meets the synthesis conditions. It is also necessary to note that the text patch cannot cover the prohibited patch area. Therefore, it is necessary to judge whether a text patch meets the preset synthesis conditions based on the patch mode and the prohibited patch area.

[0081] There are two principles to be followed when judging whether a text patch meets the synthesis conditions: Principle 1: the text patch cannot exceed the boundary of the area where the text image can be patched, because the text to be synthesized is a whole, and its corresponding text patch needs to be completely placed in the text image; Principle 2: the text patch cannot cover the prohibited patch area, that is, the location where the text patch is placed cannot cover the existing content in the text image itself;

[0082] Therefore, according to the above two principles, the boundary of the text image to be searched during pasting can be determined according to the preset pasting method to implement principle one, and principle two can be implemented according to the prohibited pasting area, thereby determining whether the text patch meets the synthesis conditions.

[0083] Step S4, when the text patch meets the synthesis condition, placing the text patch in the text image to obtain a synthesized text image.

[0084] Specifically, when the text patch meets the preset synthesis condition, the text patch is placed at the target position in the text image to complete the synthesis of the text patch and the text image.

[0085] In addition, after placement, the text patch can be cropped and random data enhancement operations can be added, such as blur, noise, compression, etc. The cropped text patch is a synthetic sample, and the text to be synthesized corresponding to the sample is the pasted text.

[0086] Based on the above method, it is possible to quickly synthesize samples with high similarity to real samples and eliminate interference from non-blank areas in the background to obtain synthetic text images for text recognition model training, thereby improving its generalization ability and recognition accuracy in various business scenarios, increasing the credibility of OCR results, and effectively saving manual labeling costs. At the same time, it improves data structuring accuracy and achieves cost reduction and efficiency improvement.

[0087] Figure 4 A schematic diagram of a synthetic text image in an exemplary embodiment of the present disclosure is schematically shown. Figure 4 As shown in FIG. 1 , multiple text patches are added and synthesized in the original text image. The height, font, etc. of these text patches are similar to the text content in the original text image, and they are all placed in the content-free area of ​​the text image.

[0088] In one embodiment of the present disclosure, in step S1, performing text detection on the text image to obtain a text font range and a text height range includes:

[0089] Step S111, performing text detection on the text image to obtain a text circumscribed rectangle;

[0090] Step S112, identifying the text font in the text circumscribed rectangle to obtain the text font range; and

[0091] Step S113, obtaining the text height range according to the height value of the text circumscribed rectangle.

[0092] refer to Figure 3 As shown, after the text image is detected, a plurality of text circumscribed rectangles corresponding to the text contents in the text image can be obtained.

[0093] Afterwards, all the text circumscribed rectangular frames are cropped, and the text font in each text strip image is identified using a font recognition method. All text font types are combined to obtain the font variation range F_range of all frames.

[0094] In addition, get the coordinates of all the text bounding rectangles, the i-th rectangle is represented as rect i, take the coordinates of the upper left corner vertex and the lower right corner vertex as rect i The coordinates [x mini ,y mini ,x maxi ,y maxi ], according to rect i Calculate the coordinates of rect i The height value H i =y maxi -y mini , get the height value H of all text circumscribed rectangles i After that, determine the minimum rectangle height h min And the maximum value of the rectangle height to obtain the text height range H_range.

[0095] It should be noted that the present disclosure does not specifically limit the execution order of step S112 and step S113, that is, the order of identifying the text font range and the text height range is not limited. Step S112 or step S113 can be executed first, or they can be executed simultaneously.

[0096] In one embodiment of the present disclosure, in step S2, obtaining the text to be synthesized, and generating a text tile corresponding to the text to be synthesized according to the text to be synthesized, the text font range, and the text height range, includes:

[0097] Step S21, randomly selecting a text to be synthesized from the corpus list;

[0098] Specifically, a text to be synthesized can be randomly selected from a corpus list of a preset corpus as the content of the text in the text patch. j It only includes the text content information, but does not include the text style, size and other information. It can be Chinese, English, numbers, punctuation marks, etc., such as "Zhuge Kongming" or "110108188801014410".

[0099] Step S22, randomly selecting a text font from the text font range;

[0100] Among them, after detecting the text image, the text font range F_range is obtained, and a text font is randomly selected from F_range. j ∈F_range. For example, bold and Song.

[0101] Step S23, randomly determining a height value from the text height range, and determining the text font size according to the height value;

[0102] After detecting the text image, we get H_range = [h min ,hmax ], randomly set a height value heght from the text height range as the height H of the text patch j , and then according to the height value H j Calculate the font size of the text in the text patch. Generally speaking, the font size can be calculated according to the formula font_size j =(H j *72) / 96 is calculated.

[0103] Step S24: generating the text patch according to the text to be synthesized, the text font and the text font size.

[0104] Among them, the content of the text j Known, font size font_size j Also known, the width of the text patch can be calculated using W j Indicates that the height of the text patch is a randomly set height value H j , based on which the size information of the text patch is obtained.

[0105] In one embodiment of the present disclosure, in step S3, judging whether the text patch meets a preset synthesis condition based on the preset patch mode and the prohibited patch area includes:

[0106] Step S31, determining a first boundary and a second boundary of the text image according to a preset pasting method, and determining a pre-placement coordinate in the text image;

[0107] Step S32, determining a pre-placement area of ​​the text patch according to the pre-placement coordinates and the size information of the text patch;

[0108] Step S33, when the pre-placement area does not exceed the first boundary and the second boundary and does not cover the forbidden patch area, returning to meet the synthesis condition;

[0109] Step S34, when the pre-placement area exceeds the first boundary, returning that the synthesis condition is not satisfied.

[0110] Specifically, before determining whether the text patch meets the synthesis condition, a preset patch mode is required, that is, the order in which the text patch is to be placed needs to be determined in advance. Different preset patch modes will determine different first boundaries, second boundaries, and pre-placement coordinates when determining whether the text patch meets the synthesis condition.

[0111] In addition, it should be noted that, generally speaking, the area that can be used for text patch synthesis is the area of ​​the entire text image, but it can also be a partial area in the text image, which is recorded as the search area. For example, the search area can be a randomly selected area of ​​a random size within the text image range, or the target area in the image can be detected by the target detection model.

[0112] When the area used for synthesis is only the search area, then the first boundary, the second boundary, and the pre-placement coordinates are determined based on the search area.

[0113] In one embodiment of the present disclosure, the pasting mode is from left to right and then from top to bottom, the first boundary is the bottom boundary, the second boundary is the right boundary, and the pre-placement coordinates are the coordinates of the upper left corner of the text image.

[0114] Specifically, since the patching method searches for a placeable area in the text image from left to right and then from top to bottom, when the text patch is synthesized, when the text patch exceeds the right boundary of the search area, the line break search can be continued, but when the text patch exceeds the lower boundary of the search area, that is, the text patch can no longer be completely placed in the search area of ​​the text image, the search can no longer be continued. Therefore, the first boundary is set as the lower boundary, and the second boundary is set as the right boundary.

[0115] The initial position where the text patch can be pasted should be the upper left corner of the search area [left, top, right, bottom] in the text image, so the upper left corner coordinates [left, top] are used as the pre-placement coordinates.

[0116] In other embodiments of the present disclosure, when the pasting method is from top to bottom and then from left to right, the first boundary is the right boundary, the second boundary is the bottom boundary, and the coordinates of the upper left corner of the search area are used as the pre-placement coordinates.

[0117] In other embodiments of the present disclosure, when the pasting mode is from right to left and then from top to bottom, the first boundary is the left boundary, the second boundary is the bottom boundary, and the coordinates of the upper right corner of the search area are used as the pre-placement coordinates.

[0118] In step S32, the pre-placement area of ​​the text tile is determined according to the pre-placement coordinates and the size information of the text tile.

[0119] Specifically, the text patch is placed according to the pre-placement coordinates. If the patch mode is from left to right, the text patch is also placed from left to right, and the coordinates of the upper left corner of the text patch are overlapped with the pre-placement coordinates, thereby obtaining the pre-placement area; if the patch mode is from right to left, the text patch is also placed from right to left. Other patch modes can be specifically set as needed, and this disclosure will not go into details here.

[0120] Given the size information of the text patch, the pre-placement area of ​​the text patch in the text image can be determined based on the pre-placement coordinates. For example, in the initial state, the pre-placement coordinates of the text image are determined to be [left, top], that is, the coordinates of the upper left corner, then the height of the text patch of the jth text to be synthesized obtained by the text image is H j , width is W j , the pre-placement area can be expressed as [left, top, left + W j ,top+H j ]. When the pre-placement coordinates are updated to [x j ,y j ], then the pre-placement area can be expressed as [x j ,y j ,x j +W j ,y j +H j ].

[0121] Step S33 and step S34 are examples of judgment results of different situations of judging whether the text tile meets the synthesis condition based on the pre-placed area.

[0122] In step S33, when the pre-placement area does not exceed the first boundary and the second boundary and does not block the prohibited tile area, it returns to satisfying the synthesis condition.

[0123] Specifically, when the text patch is placed so that it does not exceed the boundary of the text image and does not block any area with text or other content, the text patch can be placed there to perform text image synthesis.

[0124] In step S34, when the pre-placement area exceeds the first boundary, it is returned that the synthesis condition is not satisfied.

[0125] Specifically, when the text patch exceeds the first boundary of the text image after placement, it can be judged that the synthesis condition is not met. For example, if the patch method is from left to right and then from top to bottom, and the first boundary is the bottom boundary of the text image, then when the pre-placed area exceeds the bottom boundary line of the text image j_bottom, that is, the text patch can no longer fit in the text image, and it can be directly judged that the synthesis condition is not met.

[0126] Furthermore, in addition to step S33 and step S34, the method further includes:

[0127] Step S35, when the pre-placement area does not exceed the first boundary but exceeds the second boundary, updating the pre-placement coordinates in a first manner; or

[0128] Step S36, when the first boundary and the second boundary are not exceeded but the forbidden patch area is blocked, updating the pre-placement coordinates in a second manner;

[0129] Step S36, judging whether the text tile meets the synthesis condition according to the updated pre-placement coordinates.

[0130] In step S35, if the pre-placement area does not exceed the first boundary but exceeds the second boundary, there is still a chance to continue searching for the next pre-placement coordinate from the text image. The first method may be line break.

[0131] In summary, when judging whether a text patch meets the synthesis condition, only two results can be returned, one is that the synthesis condition is met, and the other is that the synthesis condition is not met. However, when judging, it can be divided into three cases, one is that the synthesis condition can be directly judged to be met, the other is that the synthesis condition can be directly judged to be not met, and the other is that it is necessary to find other pre-placed coordinates in the text image and repeat the above process to judge whether the text patch meets the synthesis condition.

[0132] Fig. 5(a) schematically shows a text patch before adjustment in an exemplary embodiment of the present disclosure. Fig. 5(b) schematically shows a text patch after adjustment in an exemplary embodiment of the present disclosure.

[0133] Referring to FIG. 5(a), 501 "Zhuge Kongming" is an added text patch, and similar ones include "Jinkou 2, Shinan District, Qingdao City, Shandong Province", "110108188801014410", and 502 "Nationality Han" and the like are text words in the text image itself, and similar ones include "name", "sex", "address", etc. The search area is the entire area 503 of the entire text image. After the text patch is placed, the pre-placement area 504 exceeds the right boundary of the text image, but does not exceed the bottom boundary, so the area where the text patch can be placed can be searched in a line break manner.

[0134] When wrapping a line, you can preset a delta yValue used to indicate the interval between two adjacent lines. When the pre-placement area exceeds the right boundary, a line break is performed and delta is added to the lower boundary of the current line in the pre-placement area. y As the upper boundary of the pre-placement area after the line break, and then continue searching from the starting position on the left.

[0135] Specifically, calculate the coordinate line of the lower boundary of the line where the current pre-placement area is located j_bottom , calculate the coordinates of the upper boundary of the next line as line j_bottom +delta y , to the starting point of the next row, thereby updating the coordinates of the pre-placement area.

[0136] Referring to FIG. 5( b ), when the pre-placement area of ​​the text patch in FIG. 5( a ) appears, a new pre-placement coordinate is determined again in a line-break manner, and after the text patch is placed, the pre-placement area is updated to 505 , completing the line-break search process.

[0137] In step S36, when the pre-placement area does not exceed the first boundary and the second boundary, but blocks the forbidden patch area, that is, the pre-placement area and the forbidden patch area have an intersection, the pre-placement coordinates need to be updated in the second way. Among them, the second way is to skip the forbidden patch area and continue searching.

[0138] Fig. 6(a) schematically shows another text patch before adjustment in an exemplary embodiment of the present disclosure. Fig. 6(b) schematically shows another text patch after adjustment in an exemplary embodiment of the present disclosure.

[0139] Referring to FIG. 6(a), similar to FIG. 5(a), 601 "Zhuge Kongming" and the like are added text stickers, 602 "Minzu Han" is the text image itself, and the search area is the entire text image 603. After the text sticker is placed, the pre-placement area 604 does not exceed the right boundary and the bottom boundary of the text image, but blocks the detected prohibited sticker area, so the text sticker can be placed over the prohibited sticker area and continue to search to the right.

[0140] When crossing the prohibited patch area, a delta can be preset x , used to indicate the minimum spacing between two adjacent areas in the same row, and to continue moving rightward by delta after crossing the forbidden patch area x The distance is then used to obtain the new pre-placement coordinates.

[0141] For example, if the rightmost edge of the text bounding rectangle that intersects the current pre-placement area is max x , then continue searching to the right so that the left boundary of the new pre-placed coordinate is maxx +delta x , and then update the pre-placement coordinates.

[0142] Referring to FIG. 6( b ), when the pre-placement area of ​​the text patch in FIG. 6( a ) appears, the search continues beyond the prohibited patch area to determine the pre-placement coordinates, and after the text patch is placed, the pre-placement area is updated to 605 .

[0143] It should be noted that there are some subtle differences in the way of crossing the forbidden patch area depending on the patch method. For example, if the patch method is from right to left, then when the pre-placement area intersects with the forbidden patch area, the leftmost edge of the current forbidden patch area needs to be max x Move left again by delta x Other updating methods also need to be determined according to the patch mode, but they all belong to the same inventive concept and fall within the protection scope of the present disclosure.

[0144] In one embodiment of the present disclosure, when determining whether a text patch meets a preset synthesis condition, different priorities may be determined according to the above determination method.

[0145] The first priority is to determine whether the pre-placed area of ​​the text patch exceeds the first boundary of the text image: if it exceeds, the synthesis is terminated directly; if it does not exceed, the second priority is to determine whether the pre-placed area exceeds the second boundary: if it exceeds the second boundary, the pre-placed coordinates are updated in the first way; if it does not exceed, the third priority is to determine whether the pre-placed area blocks the prohibited patch area: if it does not block, the placement is successful and the text patch is synthesized; if it blocks, the placement coordinates are updated in the second way. According to the above priorities, it is possible to determine whether the text patch meets the synthesis conditions at one time.

[0146] In one embodiment of the present disclosure, when a text patch meets a preset synthesis condition, the text patch is placed in a patchable area; when a text patch does not meet the preset synthesis condition, the synthesis of the text patch is terminated.

[0147] In one embodiment of the present disclosure, a plurality of text patches may be attached to a text image. Therefore, after placing the text patches in the text image, the text to be synthesized may continue to be obtained for patch synthesis.

[0148] Figure 7 The following is a schematic diagram showing a flow chart of a text image synthesis method in an exemplary embodiment of the present disclosure. Figure 7As shown, when the text patch meets the synthesis condition, after placing the text patch in the text image, the method further includes:

[0149] Step S71, updating the text image and the prohibited tile area based on the placed text tile;

[0150] Step S72, obtaining a new text to be synthesized, and generating a new text tile corresponding to the new text to be synthesized according to the new text to be synthesized, the text font range and the text height range;

[0151] Step S73, judging whether the new text patch meets the preset synthesis condition based on the preset patch mode and the updated forbidden patch area;

[0152] Step S74, when the new text patch meets the synthesis condition, placing the new text patch in the updated text image.

[0153] Specifically, after placing a text patch on a text image, the content of the text patch is added to the text image, so the text image changes, and the forbidden patch area also increases the area of ​​the placed text patch. Given the coordinates of the text patch placement and the size information of the text patch, the forbidden patch area can be updated accordingly.

[0154] Then, the text to be synthesized is obtained again, and the previous steps are repeated to generate a text patch corresponding to the text to be synthesized, and to determine whether the text patch meets the synthesis condition. This process is the same as the previous steps S2 to S4, and will not be repeated here.

[0155] In one embodiment of the present disclosure, the end of text image synthesis can also be judged. For example, after the text patch is placed on the text image, a detection step is added, which can be to detect the proportion of the current prohibited patch area to the total text image area. When the proportion is greater than a threshold, it means that the text image synthesis for the text image is ended, and at this time, no new text to be synthesized can be obtained to continue the patch synthesis process.

[0156] For example, the number of text tiles added to the text image is limited, and the text image synthesis is terminated after a preset number of text tiles are added to the text image.

[0157] The present disclosure is merely an illustrative description of this, but does not limit the present disclosure. That is, other easily conceivable solutions based on the present solution also fall within the protection scope of the present application.

[0158] Based on the above description, after generating a corresponding text patch for a text to be synthesized, it is necessary to continuously repeat the search process in the text area until a conclusion is drawn as to whether the text patch meets the synthesis condition. If the condition is met, the text patch is placed for synthesis, and if the condition is not met, the synthesis of the text patch is terminated. At the same time, it is necessary to repeatedly obtain the text to be synthesized, and repeat the previous judgment process of each text to be synthesized to synthesize the text patches corresponding to multiple texts to be synthesized.

[0159] Through the text image synthesis method provided by the present invention, the background of the real image is used, and the font range of the text in the image is identified, the height range of the circumscribed rectangular frame of the text in the image is calculated, and the background, height range, font range, and font size of the synthesized sample are determined. A more realistic synthesis is obtained to obtain a sample that is similar to the text in the real image, which can solve the problem of low similarity between the synthesized text image and the real sample.

[0160] A sliding window is used to search for blank background areas in real images, and text detection, target detection, etc. are used to obtain the coordinate positions of text, avatars, etc. in non-blank background areas in the image. It is determined whether the sliding window exceeds the sliding area and whether it intersects with the non-blank background area to obtain the background area finally used to synthesize the sample. This can solve the problem of poor quality of synthesized samples caused by interference from other characters, avatars, etc. in the extracted background during synthesis.

[0161] It allows for automatic and efficient generation of synthetic samples for text images to increase the amount of text data. These data are used for text recognition model training, which in turn can improve the generalization ability and recognition accuracy of the model, increase the credibility of OCR results, and effectively replace manual work. It is highly universal and suitable for generating printed, scanned, and electronic versions of text image synthesis.

[0162] In one embodiment of the present disclosure, the steps are specifically described by taking the synthesis of a card text image sample as an example.

[0163] (1) Traverse the card image list and randomly read an image.

[0164] (2) performing text detection on the read image;

[0165] The PSENet network based on image segmentation is used to obtain the coordinates of the external rectangular boxes of all text strips in the image, where the i-th rectangular box rect i Take the coordinates of the upper left vertex and the lower right vertex [x mini ,y mini ,x maxi ,y maxi ];

[0166] (3) Crop the rectangular frames surrounding all text strips and identify them;

[0167] Using font recognition, the font in each text strip image is obtained to obtain the font variation range of all boxes F_range = ["Fangzheng Black Simplified.ttf", "Huawen Thin Black.ttf", "OCR-B 10BT.ttf" ...];

[0168] At the same time, calculate the height of the rectangular box of all text strips in the image. i The height is H i =y maxi –y maxi , get the height range of the rectangular frame of all text bars H_range = [h min, h max ];

[0169] (4) Detect the card in the image through the object detection model, and obtain the coordinates of the card's external rectangular frame as [left, top, right, bottom]. Use this rectangular area as the text image search area that can be used for patching;

[0170] (5) Calculate text patches;

[0171] Randomly select sliding window height H j ∈[h min, h max ], according to H j And the mapping relationship between pixel value and font size, calculate the synthetic text font size font_size j =(H j *72) / 96, randomly select the text to be synthesized from the corpus of key fields of the card category j , calculate the text patch width W based on the font size and text length j , so the initial pre-placement coordinates [x j ,y j ,x j +W j ,y j +H j ], where x j =left,y j =top, randomly select text to synthesize font F j ∈F_range;

[0172] (6) Start the text patch from the starting point [left, top] in the search area [left, top, right, bottom], and follow the patching method from left to right and then from top to bottom to determine the pre-placement area [x j ,y j ,x j +Wj ,y j +H j ] situation;

[0173] (7) Calculate [x j ,y j ,x j +W j ,y j +H j ] Whether it exceeds the lower boundary of the search area:

[0174] If y j +H j >bottom, the current pre-placed area exceeds the lower boundary of the search area, which means that the patch can no longer be placed in the text image. The search is over, and the synthesis of the patch is terminated.

[0175] If y j +H j ≤bottom, then execute step (8);

[0176] (8) Calculate [x j ,y j ,x j +W j ,y j +H j ] Whether it exceeds the right boundary of the search area;

[0177] If x j +W j >right, then calculate the lower boundary line of the line where the current pre-placement area is located j_bottom , and then calculate the upper boundary of the next line as line j+1_top =line j_bottom +delta y , where delta y is the preset value, indicating the interval between two adjacent lines. The text patch is placed at the starting point of the next line, and the pre-placement area is updated to [left, line j+1_top ,left+W j ,line j+1_top +H j ], return to step (7) to continue judging;

[0178] If x j +W j ≤right, then execute step (9);

[0179] (9) Calculate [x j ,y j ,x j +W j ,y j+H j ] whether it intersects with the prohibited patch area in the text image;

[0180] If there is an intersection, the rightmost edge of the current pre-placement area is max x , continue searching to the right, so that the left boundary of the new pre-placed coordinate is max x +delta x , where delta x is the preset value, indicating the distance between the adjacent search areas on the left and right, and updates the coordinates of the pre-placed area to [max x +delta x ,y j ,max x +delta x +W j ,y j +H j ], return to step (7) to continue judging;

[0181] If there is no intersection, execute step (10);

[0182] (10) According to the selected text to be synthesized j , Font F j , font size font_size j Paste the text into the current pre-placed area, crop the area, add random data enhancement operations (such as blur, noise, compression, etc.), and the cropped text patch is a synthetic sample. The label corresponding to the synthetic sample is the pasted text text j .

[0183] (11) Determine whether the current text image synthesis is completed; if completed, synthesize the text image and all placed text patches to obtain a synthesized text image; if not completed, return to step (7).

[0184] Figure 8 The following is a schematic diagram of a card text synthesis sample in an exemplary embodiment of the present disclosure. Figure 2 As shown, it is an initial text image in one embodiment of the present disclosure, referring to Figure 4 As shown, it means Figure 2 The text image shown in the figure is synthesized, and the added text patch is placed in the real text image to obtain the added text dataset as shown in the figure. Figure 8 As shown, it includes text and image background, such as "Zhuge Kongming", "Sun Liu", etc.

[0185] By following the steps described above, you can quickly synthesize a large number of samples with a high degree of similarity to real samples for card text recognition model training, improve the accuracy of recognition of each field in the final project, and improve the model's generalization ability for card image recognition in different business scenarios.

[0186] Based on a similar method, text image synthesis can also be performed on text images such as driving licenses or medical bills. Fig. 9 A schematic diagram schematically shows a synthetic sample of a driving license text in an exemplary embodiment of the present disclosure, such as image text data such as "2000-12-01" and "Yu Yajun". Fig.10 A schematic diagram of a medical bill text synthesis sample in an exemplary embodiment of the present disclosure is schematically shown, such as image text data such as "Yueyang Hospital" and "Ammonium Chloride Licorice Mixture".

[0187] Fig.11 The following is a schematic diagram showing the composition of a text image synthesis device in an exemplary embodiment of the present disclosure. Fig.11 As shown, the text image synthesis device 1100 may include a detection module 1101, a patch module 1102, a judgment module 1103 and a synthesis module 1104. Among them:

[0188] The detection module 1101 is used to read a text image, perform text detection on the text image to obtain a text font range and a text height range, and perform content detection on the text image to obtain a prohibited patch area;

[0189] The patch module 1102 is used to obtain the text to be synthesized, and generate a text patch corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range;

[0190] A judging module 1103, configured to judge whether the text patch satisfies a preset synthesis condition based on a preset patch mode and the forbidden patch area;

[0191] The synthesis module 1104 is used to place the text patch in the text image to obtain a synthesized text image when the text patch meets the synthesis condition.

[0192] According to an exemplary embodiment of the present disclosure, the patch module 1102 is used to randomly select a text to be synthesized from a corpus list; and randomly select a text font from the text font range; and randomly determine a height value from the text height range, and determine the text font size according to the height value; and generate the text patch according to the text to be synthesized, the text font and the text font size.

[0193] According to an exemplary embodiment of the present disclosure, the judgment module 1103 is used to determine the first boundary and the second boundary of the text image in a preset patch mode, and determine a pre-placement coordinate in the text image; determine the pre-placement area of ​​the text patch according to the pre-placement coordinate and the size information of the text patch; when the pre-placement area does not exceed the first boundary and the second boundary and does not block the prohibited patch area, return that the synthesis condition is met; when the pre-placement area exceeds the first boundary, return that the synthesis condition is not met.

[0194] According to an exemplary embodiment of the present disclosure, the judgment module 1103 is also used to update the pre-placement coordinates in a first manner when the pre-placement area does not exceed the first boundary but exceeds the second boundary; or to update the pre-placement coordinates in a second manner when it does not exceed the first boundary and the second boundary but occludes the prohibited patch area; and judge whether the text patch meets the synthesis condition based on the updated pre-placement coordinates.

[0195] According to an exemplary embodiment of the present disclosure, when the pasting mode is from left to right and then from top to bottom, the first boundary is the bottom boundary, the second boundary is the right boundary, and the pre-placement coordinates are the coordinates of the upper left corner of the text image.

[0196] According to an exemplary embodiment of the present disclosure, the synthesis module 1104 also includes a loop module, which is used to update the text image and the prohibited patch area based on the placed text patch; obtain a new text to be synthesized, and generate a new text patch corresponding to the new text to be synthesized according to the new text to be synthesized, the text font range and the text height range; determine whether the new text patch meets the preset synthesis conditions based on the preset patch mode and the updated prohibited patch area; when the new text patch meets the synthesis conditions, place the new text patch in the updated text image.

[0197] According to an exemplary embodiment of the present disclosure, the detection module 1101 is used to perform text detection on the text image to obtain a text circumscribed rectangle; identify the text font in the text circumscribed rectangle to obtain the text font range; and obtain the text height range based on the height value of the text circumscribed rectangle.

[0198] The specific details of each module in the above-mentioned text image synthesis device 1100 have been described in detail in the corresponding text image synthesis method, so they will not be repeated here.

[0199] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0200] In an exemplary embodiment of the present disclosure, a storage medium capable of implementing the above method is also provided. Fig.12 A schematic diagram schematically illustrates a computer-readable storage medium in an exemplary embodiment of the present disclosure, such as Fig.12 As shown, a program product 1200 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0201] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Fig.13 A schematic diagram schematically shows the structure of a computer system of an electronic device in an exemplary embodiment of the present disclosure.

[0202] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0203] like Fig.13 As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303. In the RAM 1303, various programs and data required for system operation are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0204] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0205] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the system of the present disclosure are executed.

[0206] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0207] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0208] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0209] As another aspect, the present disclosure further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0210] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0211] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0212] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0213] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A text image synthesis method, characterized in that: include: Reading a text image, performing text detection on the text image to obtain a text font range and a text height range, and performing content detection on the text image to obtain a prohibited patch area; Acquire the text to be synthesized, and generate a text tile corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range; Determining whether the text patch meets a preset synthesis condition based on a preset patch mode and the prohibited patch area; When the text patch meets the synthesis condition, placing the text patch in the text image to obtain a synthesized text image; updating the text image and the prohibited tile area based on the placed text tile; Acquire a new text to be synthesized, and generate a new text tile corresponding to the new text to be synthesized according to the new text to be synthesized, the text font range, and the text height range; Determining whether the new text tile meets a preset synthesis condition based on a preset tile mode and the updated prohibited tile area; When the new text patch meets the synthesis condition, the new text patch is placed in the updated text image.

2. The text image synthesis method according to claim 1, characterized in that: The step of obtaining the text to be synthesized and generating a text tile corresponding to the text to be synthesized according to the text to be synthesized, the text font range, and the text height range includes: Randomly select text to be synthesized from the corpus list; and Randomly selecting a text font from the text font range; and Randomly determine a height value from the text height range, and determine the text font size according to the height value; The text tile is generated according to the text to be synthesized, the text font, and the text font size.

3. The text image synthesis method according to claim 1, characterized in that: The determining whether the text patch meets a preset synthesis condition based on the preset patch mode and the prohibited patch area includes: Determine a first boundary and a second boundary of the text image according to a preset pasting method, and determine a pre-placement coordinate in the text image; Determining a pre-placement area of ​​the text patch according to the pre-placement coordinates and size information of the text patch; When the pre-placement area does not exceed the first boundary and the second boundary and does not cover the prohibited patch area, returning to satisfy the synthesis condition; When the pre-placement area exceeds the first boundary, it is returned that the synthesis condition is not satisfied.

4. The text image synthesis method according to claim 3, characterized in that: The method further comprises: When the pre-placement area does not exceed the first boundary but exceeds the second boundary, updating the pre-placement coordinates in a first manner; or When the first boundary and the second boundary are not exceeded but the forbidden patch area is blocked, the pre-placement coordinates are updated in a second manner; It is determined whether the text tile meets the synthesis condition according to the updated pre-placement coordinates.

5. The text image synthesis method according to claim 3, characterized in that: The method further comprises: When the pasting mode is from left to right and then from top to bottom, the first boundary is the bottom boundary, the second boundary is the right boundary, and the pre-placement coordinates are the coordinates of the upper left corner of the text image.

6. The text image synthesis method according to claim 1, characterized in that: The performing text detection on the text image to obtain a text font range and a text height range includes: Performing text detection on the text image to obtain a text circumscribed rectangle; Identifying the text font in the text circumscribed rectangle to obtain the text font range; and The text height range is obtained according to the height value of the text circumscribed rectangle.

7. A text image synthesis device, characterized in that: include: A detection module, used for reading a text image, performing text detection on the text image to obtain a text font range and a text height range, and performing content detection on the text image to obtain a prohibited patch area; A patch module, used for acquiring the text to be synthesized, and generating a text patch corresponding to the text to be synthesized according to the text to be synthesized, the text font range and the text height range; A judging module, used for judging whether the text patch meets a preset synthesis condition based on a preset patch mode and the forbidden patch area; A synthesis module, used for placing the text patch in the text image to obtain a synthesized text image when the text patch meets a synthesis condition; A loop module, used for updating the text image and the prohibited patch area based on the placed text patch; obtaining a new text to be synthesized, and generating a new text patch corresponding to the new text to be synthesized according to the new text to be synthesized, the text font range and the text height range; Based on the preset patch mode and the updated prohibited patch area, it is judged whether the new text patch meets the preset synthesis condition; when the new text patch meets the synthesis condition, the new text patch is placed in the updated text image.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the text image synthesis method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the text image synthesis method as described in any one of claims 1 to 6.

Images