A method for irregular arc distortion of text images for OCR text recognition

Abstract

The present invention discloses an irregular arc distortion method for text images for OCR text recognition, which includes obtaining an original text image and an original text box annotation file; calculating the maximum arc distortion height value of the original text image; partitioning the image according to the width of the original image, generating different arc distortion height values ​​and arc wave numbers in different regions; calculating the offset distance of the image pixel points in the Y-axis direction according to the arc distortion height values ​​and arc wave numbers in different regions, obtaining new coordinates of the image pixel points, copying the pixel points to a blank image according to the new coordinates, generating an irregular arc distortion image; recalculating the original text box annotation coordinates according to the arc distortion height values ​​and arc wave numbers in different regions, generating a new multi-point annotation text box and annotation file. The present invention generates irregular arc distortion text images and corresponding annotation files, and provides a large number of training samples for OCR training arc text recognition models.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image recognition, and in particular to an irregular arc distortion method for text images for OCR text recognition. Background Art

[0002] OCR text recognition has been applied in more and more scenarios. Generally, tens of thousands of sample images are needed to train a text recognition system for different recognition scenarios. Currently, most OCR text recognition on the market is for rectangular text, and the effect of recognizing curved and distorted text is poor. In order to improve the accuracy of OCR text recognition, a large number of samples containing curved and distorted images need to be provided for OCR training models. However, in the actual acquisition process, it is difficult to collect samples of curved and distorted images, and it is time-consuming and labor-intensive to annotate text at multiple points. Summary of the invention

[0003] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a method for irregular arc distortion of text images for OCR text recognition, which automatically generates arc distortion sample images suitable for OCR arc distortion text recognition.

[0004] To achieve the purpose of the present invention, the technical solution adopted by the present invention is:

[0005] A method for irregular arc distortion of text images for OCR text recognition, comprising the steps of:

[0006] Step 1: Obtain the original text image and the original text box annotation file;

[0007] Step 2, calculating the maximum arc distortion height value of the original text image;

[0008] Step 3, partitioning the image according to the width of the original image, generating different arc distortion height values ​​and arc wave numbers in different areas;

[0009] Step 4: Calculate the offset distance of the image pixel points in the Y-axis direction according to the arc distortion height value and the number of arc waves in different areas to obtain the new coordinates of the image pixel points; generate a blank image with the same width and height as the original image, copy the pixel points to the blank image according to the new coordinates, and generate an irregular arc distortion image;

[0010] Step 5: Recalculate the original text box annotation coordinates according to the arc distortion height values ​​and arc wave numbers in different regions to generate a new multi-point annotation text box and annotation file.

[0011] Furthermore, in step 1, obtaining the original text image and the original text box annotation file includes:

[0012] Read the width Weight and height Height of the original text image;

[0013] Read the original text box annotation file, including the file name, annotation point coordinates [(x1,y1), (x2,y2), (x3,y3), (x4,y4)], that is, [text box upper left corner annotation point coordinates, text box upper right corner annotation point coordinates, text box lower right corner annotation point coordinates, text box lower left corner annotation point coordinates], and annotation text content.

[0014] Furthermore, in step 2, the maximum arc distortion height value of the original text image is calculated, including:

[0015] (2.1) Calculate the height values ​​of all text boxes in the annotation file and take the minimum height value;

[0016] (2.2) Calculate the distance between all text boxes in the annotation file and the upper boundary of the original image, and take the minimum distance value; calculate the distance between all text boxes in the annotation file and the lower boundary of the original image, and take the minimum distance value;

[0017] (2.3) Take the minimum value of the above three calculated values ​​as the maximum height value max_radian_value of the irregular arc distortion of the image.

[0018] Furthermore, in step (2.1), the coordinates of each text box annotation point are read to obtain the minimum enclosing rectangle of the text box, and the height value of the text box is obtained through its minimum enclosing rectangle, and the height value of each text box in the annotation file is calculated.

[0019] Furthermore, in step (2.2), the coordinates of the annotation points of each text box are read, where the values ​​y1 and y2 represent the distance from the text box to the upper border of the original text image, and the values ​​Height-y3 and Height-y4 represent the distance from the text box to the lower border of the original text image. The distances from each text box in the annotation file to the upper and lower borders of the original text image are calculated.

[0020] Furthermore, in step 3, different arc distortion height values ​​and arc wave numbers are generated in different areas, including:

[0021] The arc distortion height value is randomly generated in different areas, and a value random_radian_value is randomly generated between 1-max_radian_value as the arc distortion height value.

[0022] The number of arc waves is randomly generated in different areas, and the recommended value of the number of arc waves is between 1-5.

[0023] Furthermore, in step 4, the offset value of the image pixel in the Y-axis direction is calculated:

[0024] Y'=random_radian_value*sin((X_n / Width)*2*3.14*num)(1)

[0025] Among them, random_radian_value represents the arc distortion height value of the image in the area, num represents the number of arc waves, Width represents the width value of the image in the area, and X_n represents the X-axis coordinate of the image pixel point;

[0026] Then the new coordinates of the pixel point are (X_n, Y_n+Y'); where Y_n represents the Y-axis coordinate of the image pixel point.

[0027] Furthermore, in step 5, the coordinates of the original text box annotation are recalculated to generate a new multi-point annotation text box, including:

[0028] The original four-point annotation is divided into multiple annotation points according to the number of characters in the annotation text content, and the new coordinates of the divided annotation points, namely the multiple annotation coordinate points, are calculated according to the arc distortion height value and the number of arc waves; a new text box is generated according to the generated irregular arc distortion image and the new coordinates generated above, and a annotation file is generated with the new coordinates and the original annotation text content.

[0029] Furthermore, the original four-point annotation is divided into multiple-point annotations using the single-line segment point division method, including:

[0030] Connect every two adjacent annotation points into a line segment to generate four directional line segments, namely, upper left annotation point --> upper right annotation point, upper right annotation point --> lower right annotation point, lower right annotation point --> lower left annotation point, and lower left annotation point --> upper left annotation point;

[0031] Take the line segment formed by the upper left marked point --> upper right marked point, lower right marked point --> lower left marked point as a horizontal line segment, divide the line segment into multiple points, generate multiple marked points, and calculate the x value;

[0032] Take the line segment formed by the upper right marked point --> lower right marked point and the lower left marked point --> upper left marked point as a vertical line segment, divide the line segment into multiple points, generate multiple marked points, and calculate the y value;

[0033] Combine the calculated x and y values ​​one by one to generate multiple annotation points.

[0034] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method for irregular arc distortion of text images for OCR text recognition, realizes irregular image distortion and recalculation of corresponding text boxes, generates irregular arc distorted text images and corresponding annotation files, and provides a large number of training samples for OCR training arc text recognition models.

[0035] The present invention calculates the minimum height value of all text boxes, the minimum distance value from the text box to the upper boundary of the original image, and the minimum distance value from the text box to the lower boundary of the original image, and takes the minimum value of the three values ​​as the maximum height value of the maximum arc distortion of the image. The arc distortion height of each image can be adaptively adjusted to prevent the arc distortion of the image from being too large, which makes the image content unrecognizable.

[0036] The present invention divides an image into different regions according to the image width, and performs arc distortion on different regions according to different arc distortion height values, thereby producing a large number of different irregular arc distortion images.

[0037] The present invention divides the original image coordinate points into multi-point annotation coordinates according to the arc distortion height and the number of characters in the annotated text content, and generates a multi-point annotation frame that conforms to the arc distortion text after subsequent recalculation, which perfectly fits the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the irregular arc distortion method of text image for OCR text recognition according to the present invention;

[0039] Figure 2 It is a flowchart for calculating the maximum arc distortion height value of the original text image;

[0040] Figure 3 is the original annotated image provided by the embodiment of the present invention;

[0041] Figure 4 The embodiment of the present invention provides a method for dividing the original coordinate frame into multiple marking points according to the number of characters in the marking content and the distortion height;

[0042] Figure 5 It is an irregular arc-shaped distorted image provided by an embodiment of the present invention;

[0043] Figure 6 The embodiment of the present invention provides an irregular arc-shaped distorted image with an excessively large arc-shaped distorted height;

[0044] Figure 7 The irregular arc-shaped distorted image provided by the embodiment of the present invention has a text box that is too far away from the upper and lower boundaries of the image;

[0045] Figure 8 This is an irregular arc-shaped distorted image with too many arc waves provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of this application.

[0047] like Figure 1 As shown, the irregular arc distortion method of text image for OCR text recognition of the present invention comprises the following steps:

[0048] Step 1: Obtain the original text image and the corresponding original text box annotation file;

[0049] Read the width (Weight) and height (Height) of the original text image to prepare for the subsequent division of the text image into regions and calculation of the new coordinates of the image pixels.

[0050] Read the original text box annotation file and obtain all annotation data, including the file name, annotation point coordinates [(x1,y1), (x2,y2), (x3,y3), (x4,y4)], that is, [text box upper left corner annotation point coordinates, text box upper right corner annotation point coordinates, text box lower right corner annotation point coordinates, text box lower left corner annotation point coordinates], and annotation text content.

[0051] Step 2: Calculate the maximum arc distortion height value of the original text image, such as Figure 2 As shown;

[0052] (2.1) Calculate the height values ​​of all text boxes in the annotation file and take the minimum height value;

[0053] The minimum enclosing rectangle of the text box is obtained by reading the coordinates of each text box annotation point in step 1, and the height value of the text box is obtained by its minimum enclosing rectangle. The height value of each text box in the annotation file is calculated and the minimum value is taken.

[0054] The minimum value is taken because the text content in the original text image is inconsistent in size. For example, some text is size 1 and some is size 7. When the arc distortion height is too large, it has little effect on large text, but has a great impact on small text, and even distorts the small text to be unrecognizable, destroying the text content. Figure 6 Therefore, the present invention summarizes and checks the actual production results, takes the minimum text box height value for arc distortion, which can not only make large fonts arc distorted, but also make small fonts arc distorted without destroying the text content.

[0055] (2.2) Calculate the distance between all text boxes in the annotation file and the upper boundary of the original image, and take the minimum distance value; calculate the distance between all text boxes in the annotation file and the lower boundary of the original image, and take the minimum distance value;

[0056] The coordinates of the annotation points of each text box read in step 1 are used, where the values ​​y1 and y2 represent the distance from the text box to the upper boundary of the original text image, and the values ​​Height-y3 and Height-y4 represent the distance from the text box to the lower boundary of the original text image. The distances from the upper and lower boundaries of each text box in the annotation file to the original text image are calculated, and the minimum value is taken.

[0057] The minimum value is taken because when the upper or lower boundary is too small and the arc distortion height is too large, the text will be excessively offset and exceed the original text image height and disappear, such as Figure 7 Therefore, the present invention summarizes and checks the actual production results and takes the minimum boundary distance value for arc distortion, which can allow the text to be distorted in an arc shape and ensure that the text content will not exceed the original image height due to excessive arc distortion.

[0058] (2.3) Taking the minimum value among the above three calculated values ​​as the maximum height value of the irregular arc distortion of the image;

[0059] By calculating the minimum height value of the text box, the minimum distance value from the text box to the upper boundary of the original image, and the minimum distance value from the text box to the lower boundary of the original image, and taking the minimum value of the three values ​​as the maximum arc distortion height value max_radian_value of the image, the maximum arc distortion height value of each image can be adaptively calculated to avoid the situation where the image after arc distortion is too distorted, resulting in the text content being too distorted to be distinguishable, and the text content exceeds the height of the original image and is missing.

[0060] Step 3, partitioning the image according to the width of the original image, and generating an arc distortion height value and arc wave number for each area, and generating different arc distortion height values ​​and arc wave numbers for different areas;

[0061] For example, the original image is divided into two adjacent regions with the center point [Weight / 2,0] in the width direction of the original image as the dividing point, and a random arc distortion height value is generated for each region. The original image can be divided into multiple adjacent regions as required.

[0062] The method of randomly generating the arc distortion height value is to randomly generate a value random_radian_value between 1-max_radian_value. Because the random_radian_value of each area is different, the arc distortion height of each area is different, so the original image can appear irregular arc distortion.

[0063] The number of randomly generated arc waves can control the frequency of arc distortion. The recommended value is between 1 and 5. If the number of arc waves is too large, the frequency of image arc distortion will be too high, and multiple arc waves will appear, destroying the image content. Figure 8 Therefore, the present invention summarizes and checks the actual production results and takes a value of 1-5, which does not destroy the text content and can distort the image well.

[0064] Step 4: Calculate the offset distance of the image pixel points in the Y-axis direction according to the arc distortion height value and the number of arc waves in different areas, and calculate the new coordinates of the image pixel points; at the same time, generate a blank image with the same width and height as the original image, copy the pixel points to the blank image according to the new coordinates, and generate an irregular arc distortion image;

[0065] The original text image is distorted into an irregular arc. It is only necessary to calculate the offset value of each pixel point coordinate in the original image in the Y-axis direction. There is no change in the X-axis direction and no calculation is required.

[0066] The formula for calculating the offset value of the image pixel coordinates in the Y-axis direction is as follows:

[0067] Y'=random_radian_value*sin((X_n / Width)*2*3.14*num) (1)

[0068] Among them, random_radian_value represents the arc distortion height value of the image in this area, num represents the number of arc waves, Width represents the width value of the image in this area, and X_n represents the X-axis coordinate of the image pixel point.

[0069] Calculate the new Y-axis coordinate of the image pixel:

[0070] Y_n'=Y_n+Y' (2)

[0071] Wherein, Y_n represents the Y-axis coordinate of the pixel point, and Y' represents the offset distance value of the pixel point in the Y-axis direction.

[0072] Then the new coordinates of the pixel are (X_n, Y_n'). At the same time, a blank image with the same width and height as the original image is generated. When the new coordinates are calculated in the above steps, the pixel is copied to the blank image according to the new coordinates. When all the pixels in the original image are copied to the blank image, an irregular arc distorted image is generated.

[0073] Step 5: divide the original four-point annotation into multi-point annotation points according to the number of characters in the annotation text content, recalculate the original text box annotation coordinates according to the arc distortion height values ​​and the number of arc waves in different areas, and generate a new multi-point annotation text box and annotation file.

[0074] After the image is distorted into an irregular arc, the image content changes, and the original text box can no longer frame the text content of the arc distortion. The original text box needs to be recalculated as a multi-point annotation text box to completely frame the text content.

[0075] In order to convert a four-point annotation text box into a multi-point annotation text box, the present invention also designs a single line segment point method to convert a four-point annotation text box into a multi-point annotation text box. The specific method is as follows:

[0076] like Figure 3 As shown, the four-point annotation text box is composed of four annotation points. The present invention connects every two adjacent annotation points into a line segment, that is, generates four directional line segments, namely, upper left annotation point->upper right annotation point, upper right annotation point->lower right annotation point, lower right annotation point->lower left annotation point, and lower left annotation point->upper left annotation point. According to the number of characters in the annotated text content, the original text box is divided into multiple point annotation boxes.

[0077] The line segment formed by the upper left annotation point --> upper right annotation point, lower right annotation point --> lower left annotation point is taken as a horizontal line segment, and the line segment is divided into multiple points to generate multiple annotation points. The calculation method is as follows:

[0078] x=[x for x in np.linspace(x0,x1,math.ceil(len(word)+1))] (3)

[0079] Among them, x0 and x1 represent the coordinates of the two endpoints of a horizontal line segment on the X-axis respectively; len(word)+1 represents the number of annotated text characters plus 1; np.linspace(x0,x1,math.ceil(len(word)+1) means to divide the line segment between coordinates x0 and x1 into len(word)+1 points evenly.

[0080] The line segment formed by the upper right annotation point --> lower right annotation point and the lower left annotation point --> upper left annotation point is used as a vertical line segment, and the line segment is divided into multiple points to generate multiple annotation points. The calculation method is as follows:

[0081] y=[y for y in np.linspace(y0,y1,math.ceil(len(word)+1))] (4)

[0082] Among them, y0 and y1 represent the coordinates of the two endpoints of a vertical line segment on the Y axis respectively; len(word)+1 represents the number of annotated text characters plus 1; np.linspace(y0,y1,math.ceil(len(word)+1) means to divide the line segment between coordinates y0 and y1 into len(word)+1 points evenly.

[0083] Combine the calculated x and y values ​​one by one to generate multiple annotation points, such as Figure 4 shown.

[0084] The new coordinates of the divided annotation points are calculated based on the arc distortion height value and the number of arc waves, that is, all the coordinate points are put into formula (1) and formula (2) to calculate and obtain the corresponding new coordinates, that is, the multi-point annotation coordinate points. Based on the irregular arc distortion image generated in step 4 and the new coordinates generated above, a new text box is generated. The final effect is shown in the figure below. Figure 5 As shown, the new coordinates and original annotation text content are used to generate an annotation file.

[0085] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for irregular arc distortion of text images for OCR text recognition, characterized in that: Includes steps: Step 1: Obtain the original text image and the original text box annotation file; Step 2: Calculate the maximum arc distortion height value of the original text image, including: (2.1) Calculate the height values ​​of all text boxes in the annotation file and take the minimum height value; (2.2) Calculate the distance between all text boxes in the annotation file and the upper boundary of the original image, and take the minimum distance value; calculate the distance between all text boxes in the annotation file and the lower boundary of the original image, and take the minimum distance value; (2.3) Take the minimum value of the above three calculated values ​​as the maximum height value max_radian_value of the irregular arc distortion of the image; Step 3, partitioning the image according to the width of the original image, generating different arc distortion height values ​​and arc wave numbers in different areas; Step 4: Calculate the offset distance of the image pixel points in the Y-axis direction according to the arc distortion height value and the number of arc waves in different areas to obtain the new coordinates of the image pixel points; generate a blank image with the same width and height as the original image, copy the pixel points to the blank image according to the new coordinates, and generate an irregular arc distortion image; Step 5: Recalculate the original text box annotation coordinates according to the arc distortion height values ​​and arc wave numbers in different regions to generate a new multi-point annotation text box and annotation file.

2. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step 1, the original text image and the original text box annotation file are obtained, including: Read the width Weight and height Height of the original text image; Read the original text box annotation file, including the file name, annotation point coordinates [(x1,y1), (x2,y2), (x3,y3), (x4,y4)], that is, [text box upper left corner annotation point coordinates, text box upper right corner annotation point coordinates, text box lower right corner annotation point coordinates, text box lower left corner annotation point coordinates], and annotation text content.

3. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step (2.1), the coordinates of each text box annotation point are read to obtain the minimum enclosing rectangle of the text box, and the height value of the text box is obtained through its minimum enclosing rectangle, and the height value of each text box in the annotation file is calculated.

4. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step (2.2), read the coordinates of the annotation points of each text box, where the values ​​y1 and y2 represent the distance from the text box to the upper border of the original text image, and the values ​​Height-y3 and Height-y4 represent the distance from the text box to the lower border of the original text image. Calculate the distance from each text box in the annotation file to the upper and lower borders of the original text image.

5. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step 3, different arc distortion height values ​​and arc wave numbers are generated in different areas, including: The arc distortion height value is randomly generated in different areas, and a value random_radian_value is randomly generated between 1 - max_radian_value as the arc distortion height value; The number of arc waves is randomly generated in different areas, and the recommended value of the number of arc waves is between 1-5.

6. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step 4, the offset value of the image pixel in the Y-axis direction is calculated: Y' = random_radian_value * sin ((X_n / Width) * 2 * 3.14 * num) (1) Among them, random_radian_value represents the arc distortion height value of the image in the area, num represents the number of arc waves, Width represents the width value of the image in the area, and X_n represents the X-axis coordinate of the image pixel point; Then the new coordinates of the pixel point are (X_n, Y_n + Y'); where Y_n represents the Y-axis coordinate of the image pixel point.

7. The irregular arc distortion method for text images for OCR text recognition according to claim 1, characterized in that: In step 5, the coordinates of the original text box annotation are recalculated to generate a new multi-point annotation text box, including: The original four-point annotation is divided into multiple annotation points according to the number of characters in the annotation text content, and the new coordinates of the divided annotation points, namely the multiple annotation coordinate points, are calculated according to the arc distortion height value and the number of arc waves; a new text box is generated according to the generated irregular arc distortion image and the new coordinates generated above, and a annotation file is generated with the new coordinates and the original annotation text content.

8. The method for irregular arc distortion of text images for OCR character recognition according to claim 7, characterized in that: The original four-point annotation is divided into multiple-point annotations using the single-line segment point division method, including: Connect every two adjacent annotation points into a line segment to generate four directional line segments, namely, upper left annotation point --> upper right annotation point, upper right annotation point --> lower right annotation point, lower right annotation point --> lower left annotation point, and lower left annotation point --> upper left annotation point; Take the line segment formed by the upper left marked point --> upper right marked point, lower right marked point --> lower left marked point as a horizontal line segment, divide the line segment into multiple points, generate multiple marked points, and calculate the x value; Take the line segment formed by the upper right marked point --> lower right marked point and the lower left marked point --> upper left marked point as a vertical line segment, divide the line segment into multiple points, generate multiple marked points, and calculate the y value; Combine the calculated x and y values ​​one by one to generate multiple annotation points.

Images