A color feature-sensitive intelligent text synthesis method

Abstract

This invention belongs to the field of deep learning in artificial intelligence. For the problem of model data thirst in deep learning, synthetic data can effectively make up for the shortage of training data and save time, manpower and money. In text detection and recognition tasks, it is currently popular to adopt the strategy of "synthetic data training model, real data verification model". Therefore, the quality of synthetic data directly affects the performance of the detector or recognizer. Based on this, a novel text synthesis technology is proposed, which finds the most suitable color for the current text according to the color distribution law of the background image, thereby avoiding the problem of experimental results affected by the similar color of the text and the background. The invention can be applied to text detection and recognition tasks, and can improve the performance of detectors and recognizers.

Description

Technical Field

[0001] This invention belongs to the field of deep learning in artificial intelligence. For the problem of model data thirst in deep learning, synthetic data can effectively make up for the shortage of training data and save time, manpower and money. In text detection and recognition tasks, it is currently popular to adopt the strategy of "synthetic data training model, real data verification model". Therefore, the quality of synthetic data directly affects the performance of the detector or recognizer. Based on this, a novel text synthesis technology is proposed, which finds the most suitable color for the current text according to the color distribution law of the background image, thereby avoiding the problem of experimental results being affected by the similar color of the text and the background. The invention can be applied to text detection and recognition tasks, and can improve the performance of detectors and recognizers. Background Art

[0002] With the rapid development of deep learning, the model's demand for data has increased. The emergence of synthetic data has solved this traditional problem, but how to improve the quality of synthetic data to support the model to conduct more efficient research is still a problem that needs to be solved urgently.

[0003] In the task of text detection and recognition, it can be divided into two independent research contents, namely detection and recognition.

[0004] First, in the text recognition task, the relevant synthesis technology mainly synthesizes text through steps such as font rendering, shadow rendering, basic coloring, perspective transformation, and adding noise. After this series of operations, a corresponding text instance image can be obtained, and relevant experiments have proved that this method can be efficiently applied to model training tasks.

[0005] Secondly, in the text detection task, since the above-mentioned synthesis method only generates an instance image containing a text, it is not suitable for detecting content. In response to the complex and difficult text detection problem, some researchers have proposed an innovative generation method based on text position, which limits the text to the same depth of the image. Specifically, the most suitable position for text placement is selected based on the screening results of semantic information and depth. Some researchers have proposed a method based on semantic consistency, which believes that in the process of synthesizing pictures, some discordant text generation areas, such as faces, should be filtered out. On this basis, the final text generation will also go through two processes: saliency analysis and optimal text position. The former mainly helps to embed the text into the correct position in the semantically reasonable area. The latter adaptively determines the color and brightness of the source text according to the color and brightness of the background image around the embedding position, and can also adaptively determine the direction of the source text according to the direction of the context structure around the embedding position. In addition, some researchers have proposed a method based on the UE4 three-dimensional engine to build a virtual world to imitate the real world. Its biggest feature is to render the text instance together with the background image to minimize the difference between the text and the background. This method can also generate multi-language versions.

[0006] Similar to this is the Generative Adversarial Network (GAN), which has also shown great potential in synthesizing images.

[0007] Since synthesis methods have achieved excellent results in recognition tasks, existing related work has focused on how to improve the accuracy of text detection. However, this type of method ignores a key link in synthesizing text - text coloring. In contrast, this paper analyzes the irrationality of existing text coloring schemes, while considering the randomness of the possible locations where text may appear. On this premise, a new intelligent text generation algorithm is proposed, which can better solve the above problems and improve the performance of the model. Summary of the invention

[0008] (1) Core idea

[0009] Based on the text synthesis technology that is sensitive to color features, first of all, the colors used for coloring have the principle that similar colors are inseparable, that is, when the two selected colors are within a certain range, the two will be difficult to distinguish. Specifically, taking grayscale pixels as an example, there are a total of 256 grayscale values. We know through experiments and related data that when the pixel range value is x, the pixels can only be barely recognizable when they are within the range of x. For example, when the target pixel is the 50th pixel, the pixels in the range of x before and after it have interference effects. Therefore, only when the obtained pixel is less than or equal to the (50-x-1)th pixel or greater than or equal to the (50+x+1)th pixel, can a relatively clear and recognizable optional color pixel be obtained. For the three primary colors, their characteristics are consistent with grayscale colors and follow the same color rules. Secondly, in reality, the location of text is random. When the angle of view is different, the text may appear half in the sky and half on the building. At the same time, there are many text images with complex backgrounds in real scenes, such as promotional pictures. For this type of problem, existing methods are not applicable because of the unreasonable color schemes they adopt. In contrast, the advantage of our method is that it is based on color features, so it can effectively solve such problems. In summary, the proposed method can solve problems that are currently ignored by most technologies, thereby improving the performance of the model.

[0010] (2) Algorithm description

[0011] The general process of the entire algorithm is as follows Figure 2 As shown, first randomly select a position from the input image to place

[0012] The text is placed; then the background pixels around the text are obtained through the expansion operation (the text is expanded outward twice); then the number of characters in the current synthesis task and the number of background pixels just obtained are counted; then, the two key steps in this method are entered, and the two thresholds T1 and T2 are set for screening; finally, a pixel is randomly selected from the remaining pixel set to color the text.

[0013] The following is a more detailed description of each step:

[0014] First, the algorithm inputs a random position in the input image to place the text, and then obtains the pixels after the text in the current image is expanded by two circles (2 pixels) (referred to as the impact factor), and then performs some statistical operations, including counting the number of characters in the synthesized image, the number of extracted pixels, etc.

[0015] Secondly, the most critical part of the entire synthesis method is the two thresholds of the algorithm. In the process of selecting the first threshold (T1), "(0.1 / word_count)*length_dic" is used here (here word_count and length_dic represent the number of characters and the number of pixels respectively). Specifically, 0.1 represents 1 / 10 of a pixel. Here, referring to the dot above the "i" in the English letter, the dot is the smallest unit of pixel and accounts for approximately 1 / 10 of the whole. Therefore, the coefficient here is 0.1. Then divide 0.1 by the total number of characters to get the smallest unit pixel value of all font pixels. Finally, multiply it by the total number of pixels to get a specific threshold (T1).

[0016] After obtaining the first threshold (T1), we start with the first pixel point and then perform breadth-first traversal in the "8" direction to screen available pixels. The specific rules of traversal are as follows: first record the starting pixel value (unchanged), then traverse in 8 directions (upper left, upper, upper right, right, lower right, lower, lower left, left). Here is a visualization of the breadth-first traversal in 8 directions. Figure 4 If the difference between the traversed pixel value and the starting pixel value is less than or equal to n, then add it to the candidate color list, otherwise delete it. And so on, until all pixels are traversed.

[0017] Figure 3 The first threshold processing process is shown in the form of a flow chart.

[0018] After obtaining the candidate color list through the above steps, the algorithm will face the second screening. The threshold used in the screening here refers to the color-sensitive feature point, that is, x is used as the second threshold T2 (the relevant content has been explained in the "Core Idea"). The specific screening process is: all background pixels that are not selected as candidate color lists are screened within the range of x. Pixels within this range and in the candidate color list will be deleted, and finally the pixels that still exist in this list will be upgraded to available pixels. Finally, the algorithm will randomly select one of the available pixels for text coloring.

[0019] It should be noted that if there are no available pixels after filtering within the x range, the algorithm will automatically return to the first step, that is, reselect a text position and then perform the same operation. If there are still no available pixels after 20 iterations, the synthesis will be abandoned.

[0020] The second threshold T2 screening process can be found in detail. Figure 5 .

[0021] Finally, after screening by two thresholds T1 and T2, a final set of available pixels can be obtained, and then a pixel is randomly selected from the pixel set as the color pixel for text rendering. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Each detailed step of the text synthesis method is presented in a graphical manner.

[0023] Figure 2 The synthetic steps of the synthetic method are summarized in the form of a flow chart.

[0024] Figure 3 The working principle of the first threshold value T1 is shown in the form of a flow chart.

[0025] Figure 4 This is a visualization image of the 8-direction breadth-first traversal method used during the first threshold T1 processing.

[0026] Figure 5 The working principle of the second threshold value T2 is shown in the form of a flow chart.

[0027] Figure 6 Some samples generated by this method show that the text and the background are clearly visible, and the text is clearly visible without any color confusion.

[0028] Algorithm 1 Intelligent text synthesis algorithm (ITC) based on color features

[0029] Input: dictionary index (index_dic, used to extract all pixels around the text), total word count (word_count, counts the number of words in the synthetic image), number of pixels (length_dic, extracts the number of all affected pixels), filter threshold (group_thresh, where 0.1 is used as 1 / 10 pixel of a single word, then divided by the total number of words, and finally multiplied by the total number of extracted pixels to get the final threshold)

[0030] Output: List of available colors

[0031]

Claims

1. A color feature-sensitive intelligent text synthesis method, characterized in that: The method comprises the following steps: First, obtain the background image that needs to be synthesized with text, and determine whether there is text in the image. If there is text in the image, abandon this image and select the next image, and then determine again until a picture without text is selected as input; If there is no other text in the image, use this image as input and conduct the experiment; After selecting the background image, a position for generating text is randomly selected in the image, and its size is consistent with the size of the pre-selected text; After the text is placed in the selected position in the picture, the specific steps are: Expand the text outward by two circles of pixels to obtain the background pixels within the two-pixel range around the text; Count the number of characters and the number of background pixels obtained; Set the first threshold and filter the acquired background pixels; Set a second threshold and perform a second filter on the background pixels filtered by the first threshold; Use the available pixels after secondary filtering for text coloring; After obtaining the background pixels, set the first threshold to filter out some interference pixels. The specific steps are: First, the first threshold is set to T1 to determine whether there is a background pixel. If not, the filtering process of the first threshold is terminated directly and the filtering process of the second threshold T2 is entered. If there are background pixels, the filtering stage begins. The detailed steps of the filtering stage are as follows: Select a pixel of the first character as the starting point and add it to the candidate list; Starting from the starting pixel, perform a breadth-first traversal of the background pixels in 8 directions; Compare the difference between each pixel value traversed and the starting pixel value. If it meets the set conditions, it will be added to the candidate list, otherwise it will be deleted; After the traversal is completed, all pixels added to the candidate list are filtered by the first threshold T1, and the filtered pixels that meet the conditions are passed to the second threshold T2; After obtaining the candidate list filtered by the first threshold T1, it is filtered by the second threshold T2. The specific steps are as follows: First, get all the filtered candidate list pixels; Since there are 256 pixel values ​​in total for grayscale pixels, obtain the pixels that are not already in the candidate list - unused pixels; Perform range screening on unused pixels, use the second threshold T2 as the screening range, filter each unused pixel in turn, that is, "current unused pixel value ± T2", and compare the pixels in the candidate list in turn. If there are pixels in the candidate list within the range of "current unused pixel value ± T2", delete them until all unused pixels are compared. Finally, the pixels in the candidate list after screening are automatically upgraded to the available pixel list; If there are no available pixels after filtering, the whole process returns to the first step, that is, reselecting a position for text generation and repeating the operation.

2. The method according to claim 1, characterized in that After the step of determining whether there is text in the background image, the following steps are further included: Record the name of the current background image used as input and the coordinate information of the text generation position.

3. The method according to claim 1, characterized in that After selecting the starting pixel, perform a breadth-first traversal in 8 directions. The specific steps of the traversal are: After selecting the starting point, traverse in the order of upper left -> upper -> upper right -> right -> lower right -> lower -> lower left -> left.

4. The method according to claim 1, characterized in that In the process of 8-direction breadth-first traversal, difference comparison is used, and pixels that meet certain conditions will be added to the candidate list. The specific steps are as follows: During the traversal process, the difference between the current pixel value traversed and the starting pixel value is compared. When the absolute value of the difference between the two pixel values ​​is less than n, it is added to the candidate list and the next traversal is performed. Otherwise, the next traversal is directly performed.

5. The method according to claim 1, characterized in that After the traversal is completed, a candidate list containing multiple background pixels can be obtained, and then the candidate list is filtered by the first threshold T1. The specific steps of the filtering are: According to the set threshold value T1, each pixel value in the candidate list is compared in turn, all pixel values ​​less than the threshold value are deleted from the candidate list, and the filtered candidate list is passed to the second threshold value T2 for filtering.

6. The method according to claim 1, characterized in that After obtaining the list of available pixels, the current text can be colored. The coloring steps are: A random selection strategy is adopted for the obtained list of available pixels and the text is colored.

Images