A method, system and storage medium for removing overlapping of image-type text line detection areas

Abstract

The present invention relates to the field of document processing, and discloses a method, system and storage medium for removing overlapping of image-type text line detection areas. The system for removing overlapping of image-type text line detection areas is loaded on a local GPU server, image-type text is uploaded, and a text line detection algorithm based on deep learning and computer vision technology is used to detect and locate image-type text line areas. The overlapping areas of the same line of the image-type text and the overlapping areas of the high-order line and the low-order line are judged, the overlapping areas are removed, the coordinate information of the overlapping areas is recalculated and updated, and the coordinate information of the processed text areas is output. When text recognition is performed on all text areas in the subsequent text recognition, repeated recognition of text content is avoided.

Description

Technical Field

[0001] The present invention relates to the field of document processing, and in particular to a method, system and storage medium for removing overlap of image-type text line detection areas. Background Art

[0002] With the development of smart phones and the Internet, it is easier and faster to obtain and share image-based text. The text in the scene text contains rich and accurate information. The detection of text information in the text is of great significance for the understanding of the text content. At present, most of the processing methods for the acquired text are: segmenting the text area from the background, and processing it using subsequent text recognition, translation, voice broadcast and other methods. However, there are challenges in text detection such as complex background, diverse text expressions, overlapping text content, high technical difficulty and poor text quality. Traditional processing methods can no longer meet the requirements of solving these problems. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention provides a method, system and storage medium for removing overlap of image-type text line detection areas.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] A method for removing overlap of image-based text line detection areas comprises the following steps:

[0006] S1: upload image-based text;

[0007] S2: Use a text line detection algorithm based on deep learning and computer vision technology to detect and locate image-based text line areas. Each text line area is represented by a long rectangular block. The coordinate information of the four vertices of the text line area is calculated and marked as upper left, upper right, lower left, and lower right respectively.

[0008] S3: sorting the row area block coordinate information according to the upper left and lower left coordinates of the text row area, wherein the long rectangular blocks belonging to the same row are arranged together and marked as belonging to the same row;

[0009] S4: Analyze all text line areas in the same line to determine whether there is an overlapping area;

[0010] S5: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the upper right and lower right coordinate information of the previous text line area of ​​the overlapping area to recalculate and update the upper left and lower left vertex coordinate information of the next overlapping text line area;

[0011] S6: analyzing the text line area of ​​the adjacent high and low order lines, and determining whether there is an overlapping area between the high order line and the low order line by calculating the vertex coordinate information of the lower left and lower right of the high order line and the vertex coordinate information of the upper left and upper right of the low order line;

[0012] S7: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the lower left and lower right coordinate information of the high-order line to recalculate and update the upper left and upper right vertex coordinate information of the overlapping low-order line;

[0013] S8: Analyze and update the coordinate information of all the overlapping text line areas, and output the coordinate information of all the text line area blocks without the overlapping areas.

[0014] Furthermore, the step S1 includes: executing a file encryption program when uploading image-type text.

[0015] Furthermore, the step S2 includes: realizing text area detection and area positioning and marking it in the form of a long rectangle, analyzing and calculating the coordinate information of the four vertices of the rectangle, and the coordinate position is in pixels.

[0016] Furthermore, the step S3 includes: sorting the row areas by calculating the row height values ​​of the upper left and lower left of the text row areas, and listing the text row areas belonging to the same row as one row.

[0017] Furthermore, the steps S4 and S5 include:

[0018] First, by calculating the coordinate information of the four vertices of the long rectangular blocks of the left and right adjacent text line areas, the width pixels and height pixels are calculated, and the coordinate information of the four vertices of the left text line area is obtained counterclockwise. Get the coordinate information of the four vertices of the right text line area counterclockwise

[0019] Secondly, the coordinate information of the upper left vertex of the left text line area Upper right vertex coordinate information And the coordinate information of the upper left vertex of the right text line area Upper right vertex coordinate information Substitute the width in pixels into the following formula:

[0020]

[0021] Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area in the text line area of ​​the same line. According to the coordinate information of the upper right vertex of the left text line area Coordinate information of the lower right vertex Width pixels, recalculate and update the coordinate information of the upper left vertex of the right text line area Coordinate information of the lower left vertex The width of the two vertex coordinates in pixels; otherwise, there is no overlapping area in the text line area of ​​the same line.

[0022] Furthermore, the steps S6 and S7 include:

[0023] First, by calculating the coordinate information of the four vertices in the high-order row and the low-order row, the width pixel and the height pixel are calculated, and the coordinate information of the four vertices in the high-order row is obtained counterclockwise. And counterclockwise to get the coordinate information of the four vertices in the low row

[0024] Secondly, the coordinate information of the lower left vertex of the high-order row Coordinate information of the lower right vertex And the coordinate information of the upper left vertex of the low-order row Upper right vertex coordinate information Substitute the height pixels into the following formula:

[0025]

[0026] Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area between the high-order row and the low-order row. According to the coordinate information of the lower left vertex of the high-order row, Coordinate information of the lower right vertex The height pixel is recalculated and updated to the upper left vertex coordinate information of the low-order text line area Upper right vertex coordinate information The height pixels of the two vertex coordinates; otherwise, there is no overlapping area between the high-order row and the low-order row.

[0027] Furthermore, the image-based text is a PDF scan of a picture or text.

[0028] A system for removing overlapping of image-type text line detection areas, the system comprising:

[0029] A memory for storing executable instructions;

[0030] The processor is used to implement any of the above-mentioned methods for removing overlap of image-type text line detection areas when running the executable instructions stored in the memory.

[0031] A computer-readable storage medium stores executable instructions, which, when executed by a processor, implement any of the above-mentioned methods for removing overlap of image-type text line detection areas.

[0032] The present invention provides a method, system and storage medium for removing overlaps in image-type text line detection areas. The method detects and locates image-type text line areas through a text line detection algorithm based on deep learning and computer vision technology, obtains coordinate information of the line area, and then obtains overlapping areas through calculation and analysis, recalculates and updates the coordinate information of the overlapping areas, and realizes text area text recognition without duplication of text content. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of an embodiment of the present invention.

[0034] Figure 2 This is an uploaded text diagram according to an embodiment of the present invention.

[0035] Figure 3 FIG. 4 is a schematic diagram of row area overlap according to an embodiment of the present invention.

[0036] Figure 4 FIG. 4 is a schematic diagram of the overlap of the high-order row and the low-order row according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Combination Figure 1-2 As shown, a method for removing overlapping of image-based text line detection areas comprises the following steps:

[0039] S1: upload image-based text;

[0040] Execute file encryption when uploading image-type text.

[0041] S2: Use a text line detection algorithm based on deep learning and computer vision technology to detect and locate image-based text line areas. Each text line area is represented by a long rectangular block. The coordinate information of the four vertices of the text line area is calculated and marked as upper left, upper right, lower left, and lower right respectively.

[0042] Each long rectangular block refers to a text line area block, and each line of text includes one or more line area blocks;

[0043] Realize text area detection and area positioning and mark it in the form of a long rectangular block, analyze and calculate the coordinate information of the four vertices of the rectangular block. The coordinate position is in pixels, height is expressed as height pixels, and width is expressed as width pixels;

[0044] S3: sorting the row area block coordinate information according to the upper left and lower left coordinates of the text row area, wherein the long rectangular blocks belonging to the same row are arranged together and marked as belonging to the same row;

[0045] The row areas are sorted by calculating the row height values ​​of the upper left and lower left of the text row area, and the text row areas belonging to the same row are listed as one row.

[0046] S4: Analyze all text line areas in the same line to determine whether there is an overlapping area;

[0047] S5: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the upper right and lower right coordinate information of the previous text line area of ​​the overlapping area to recalculate and update the upper left and lower left vertex coordinate information of the next overlapping text line area;

[0048] By calculating the coordinate information of the four vertices of the long rectangular blocks on the same row, it is determined whether there is an overlapping area between the left and right adjacent long rectangular blocks. If there is an overlapping area, the position and size of the left and right overlapping areas are calculated, and the coordinate information of the upper left and lower left vertices of the long rectangular block on the right is recalculated and updated.

[0049] For example Figure 3 As shown, there are two adjacent and overlapping long rectangular blocks in the same row, and the four vertex coordinate information of the left long rectangular block in the same row is marked as The coordinate information of the four vertices of the right long rectangular block is marked as

[0050] First, by calculating the coordinate information of the four vertices of the long rectangular blocks of the left and right adjacent text line areas, the width pixels and height pixels are calculated, and the coordinate information of the four vertices of the left text line area is obtained counterclockwise. Get the coordinate information of the four vertices of the right text line area counterclockwise

[0051] Secondly, the coordinate information of the upper left vertex of the left text line area Upper right vertex coordinate information And the coordinate information of the upper left vertex of the right text line area Upper right vertex coordinate information Substitute the width in pixels into the following formula (I):

[0052]

[0053] Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area in the text line area of ​​the same line. According to the coordinate information of the upper right vertex of the left text line area Coordinate information of the lower right vertex Width pixels, recalculate and update the coordinate information of the upper left vertex of the right text line area Coordinate information of the lower left vertex The width of the two vertex coordinates in pixels;

[0054] On the contrary, if the result calculated by substituting var into the formula is var<0.01, there is no overlapping area in the text line area of ​​the same line.

[0055] S6: analyzing the text line area of ​​the adjacent high and low order lines, and determining whether there is an overlapping area between the high order line and the low order line by calculating the vertex coordinate information of the lower left and lower right of the high order line and the vertex coordinate information of the upper left and upper right of the low order line;

[0056] S7: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the lower left and lower right coordinate information of the high-order line to recalculate and update the upper left and upper right vertex coordinate information of the overlapping low-order line;

[0057] By calculating the coordinate information of the four vertices of the long rectangular blocks of the adjacent high and low rows, it is determined whether there is an overlapping area between the long rectangular blocks of the high and low rows. If there is an overlapping area, the position and size of the upper and lower overlapping areas are calculated, and the coordinate information of the upper left and upper right vertices of the long rectangular block of the low row is recalculated and updated.

[0058] For example Figure 4 As shown, the high-order row and the low-order row have overlapping long rectangular blocks, and the coordinate information of the four vertices in the high-order row is marked as The coordinates of the four vertices in the low row are marked as

[0059] First, by calculating the coordinate information of the four vertices in the high-order row and the low-order row, the width pixel and the height pixel are calculated, and the coordinate information of the four vertices in the high-order row is obtained counterclockwise. And counterclockwise to get the coordinate information of the four vertices in the low row

[0060] Secondly, the coordinate information of the lower left vertex of the high-order row Coordinate information of the lower right vertex And the coordinate information of the upper left vertex of the low-order row Upper right vertex coordinate information Substitute the height pixel into the following formula (II):

[0061]

[0062] Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area between the high-order row and the low-order row. According to the coordinate information of the lower left vertex of the high-order row, Coordinate information of the lower right vertex The height pixel is recalculated and updated to the upper left vertex coordinate information of the low-order text line area Upper right vertex coordinate information The height pixels of the two vertex coordinates;

[0063] On the contrary, if the result calculated by substituting var into the formula is var<0.01, there is no overlapping area between the high-order row and the low-order row.

[0064] S8: Analyze and update the coordinate information of all the overlapping text line areas, and output the coordinate information of all the text line area blocks without the overlapping areas.

[0065] The image-based text in the present invention refers to a PDF scan of a picture or text.

[0066] The present invention provides a system for removing overlapping of image-type text line detection areas, the system comprising:

[0067] A memory for storing executable instructions;

[0068] The processor is used to implement any of the above-mentioned methods for removing overlap of image-type text line detection areas when running the executable instructions stored in the memory.

[0069] The system for removing overlapping of image-type text line detection areas provided by the present invention can be used by multiple users concurrently, and during the use, file upload is kept confidential.

[0070] The present invention provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement any of the above-mentioned methods for removing overlap of image-type text line detection areas.

[0071] In combination with the present invention Figure 1-4As shown, by loading a system for removing overlapping image-type text line detection areas on a local GPU server, the system is used by multiple users concurrently. Users upload image-type texts, and use a text line detection algorithm based on deep learning and computer vision technology to detect and locate the image-type text line area. The four vertex coordinate information of the text line area is calculated and analyzed. According to the coordinate information of all text line areas, the line areas are sorted, and the text line area blocks belonging to the same line are marked as the same line. It is analyzed whether there are overlapping areas in all text line areas of the same line. First, the above formula (I) is used to calculate whether there are overlapping areas in the left and right adjacent text line areas. Overlapping area, when the calculation result is var≥0.01, it can be determined that there is an overlapping area in the text line area of ​​the same line, analyze and calculate the position and size of the overlapping area, recalculate and update the upper left and lower left vertex coordinate information of the right text line area, secondly, determine whether there is an overlapping area between the high-order line and the low-order line, and use the above formula (II) to calculate whether there is an overlapping area between the text line areas of the high-order line and the low-order line. When the calculation result is var≥0.01, it can be determined that there is an overlapping area between the high-order line and the low-order line, analyze and calculate the position and size of the text line overlapping area, and update the upper left and upper right vertex coordinate information of the low-order line.

[0072] The above formula and value range are used to determine whether there is an overlapping area in the image text. When there are no less than 3 text line area blocks in the same row, the algorithm for detecting the overlapping area is consistent with the above formula (I). From left to right, it is determined whether there is an overlapping area in the left and right adjacent text line areas. If there is an overlapping area, it is removed until all the overlapping areas of the multiple text line area blocks in the same row are completely removed. For example, there are three text line area blocks in the same row, which are called A, B, and C text line area blocks from left to right. When determining whether there is an overlapping area, first determine the A and B text line area blocks, and use the above formula (I) to determine. If there is an overlapping area, recalculate and update the upper left and lower left vertex coordinate information of the B text line area block. If not, skip it; then determine the B and C text line area blocks, and also use the above formula (I) to determine. If there is an overlapping area, recalculate and update the upper left and lower left vertex coordinate information of the C text line area block. If there is no overlapping area, no change is required. If there are more than three text row area blocks in the same row, the method for detecting and removing overlapping areas is the same as the method for detecting and removing overlapping areas in the same row.

[0073] When there are no less than 3 lines per page of image-type text, the algorithm for detecting overlapping areas is consistent with the above formula (II). From top to bottom, it is determined whether there are overlapping areas in the text line areas of the high-order lines and low-order lines. If there are overlapping areas, they are removed until the overlapping areas of the text line area blocks of multiple lines are completely removed. For example, there are three lines of text line area blocks, which are called A', B', and C' text line area blocks from top to bottom. When determining whether there are overlapping areas, first determine the A' and B' text line area blocks and use the above formula (II) to determine. If there are overlapping areas, the upper left and upper right vertex coordinate information of the B' text line area block is recalculated and updated. If not, it is skipped; secondly, determine the B' and C' text line area blocks and use the above formula (II) to determine. If there are overlapping areas, the upper left and upper right vertex coordinate information of the C' text line area block is recalculated and updated. If there are no overlapping areas, no changes are made. If there are more than three text line area blocks on the same page of text, the method for detecting and removing overlapping areas is the same as the method for the three text line area blocks mentioned above.

[0074] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for removing overlapping of image-based text line detection areas, characterized in that: The following steps are involved: S1: upload image-based text; S2: Use a text line detection algorithm based on deep learning and computer vision technology to detect and locate image-based text line areas. Each text line area is represented by a long rectangular block. The coordinate information of the four vertices of the text line area is calculated and marked as upper left, upper right, lower left, and lower right respectively. S3: sorting the row area block coordinate information according to the upper left and lower left coordinates of the text row area, wherein the long rectangular blocks belonging to the same row are arranged together and marked as belonging to the same row; S4: Analyze all text line areas in the same line to determine whether there is an overlapping area; S5: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the upper right and lower right coordinate information of the previous text line area of ​​the overlapping area to recalculate and update the upper left and lower left vertex coordinate information of the next overlapping text line area; S6: analyzing the text line area of ​​the adjacent high and low order lines, and determining whether there is an overlapping area between the high order line and the low order line by calculating the vertex coordinate information of the lower left and lower right of the high order line and the vertex coordinate information of the upper left and upper right of the low order line; S7: Analyze the text line area with overlapping area and calculate the position and size of the overlapping area, and use the lower left and lower right coordinate information of the high-order line to recalculate and update the upper left and upper right vertex coordinate information of the overlapping low-order line; S8: Analyze and update the coordinate information of all the overlapping text line areas, and output the coordinate information of all the text line area blocks without the overlapping areas.

2. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: Step S1 includes: executing a file encryption program when uploading image-type text.

3. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: The step S2 includes: realizing text area detection and area positioning and marking it in the form of a long rectangle, analyzing and calculating the coordinate information of the four vertices of the rectangle, and the coordinate position is in pixels.

4. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: The step S3 includes: sorting the row areas by calculating the row height values ​​of the upper left and lower left of the text row areas, and listing the text row areas belonging to the same row as one row.

5. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: The steps S4 and S5 are specifically as follows: First, by calculating the coordinate information of the four vertices of the long rectangular blocks of the left and right adjacent text line areas, the width pixels and height pixels are calculated, and the coordinate information of the four vertices of the left text line area is obtained counterclockwise. Get the coordinate information of the four vertices of the right text line area counterclockwise Secondly, the coordinate information of the upper left vertex of the left text line area Upper right vertex coordinate information And the coordinate information of the upper left vertex of the right text line area Upper right vertex coordinate information Substitute the width in pixels into the following formula: Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area in the text line area of ​​the same line. According to the coordinate information of the upper right vertex of the left text line area Coordinate information of the lower right vertex Width pixels, recalculate and update the coordinate information of the upper left vertex of the right text line area Coordinate information of the lower left vertex The width of the two vertex coordinates in pixels; otherwise, there is no overlapping area in the text line area of ​​the same line.

6. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: The steps S6 and S7 are specifically as follows: First, by calculating the coordinate information of the four vertices in the high-order and low-order rows, the broadband pixel and the height pixel are calculated, and the coordinate information of the four vertices in the high-order row is obtained counterclockwise. And counterclockwise to get the coordinate information of the four vertices in the low row Secondly, the coordinate information of the lower left vertex of the high-order row Coordinate information of the lower right vertex And the coordinate information of the upper left vertex of the low-order row Upper right vertex coordinate information Substitute the height pixels into the following formula: Finally, when the calculation result is var ≥ 0.01, it is determined that there is an overlapping area between the high-order row and the low-order row. According to the coordinate information of the lower left vertex of the high-order row, Coordinate information of the lower right vertex The height pixel is recalculated and updated to the upper left vertex coordinate information of the low-order text line area Upper right vertex coordinate information The height pixels of the two vertex coordinates; otherwise, there is no overlapping area between the high-order row and the low-order row.

7. The method for removing overlapping of image-based text line detection areas according to claim 1, characterized in that: The image-type text is a PDF scan of a picture or text.

8. A system for removing overlapping detection areas of image-based text lines, characterized in that: The system for removing overlapping of image-based text line detection areas comprises: A memory for storing executable instructions; The processor is used to implement the method for removing overlap of image-type text line detection areas as described in any one of claims 1 to 7 when running the executable instructions stored in the memory.

9. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by the processor, the method for removing overlap of image-type text line detection areas as described in any one of claims 1 to 7 is implemented.

Images