A Bill Text Detection Method Incorporating Prior Knowledge of Character Color

Abstract

The present invention discloses a bill text detection method that integrates the prior knowledge of text color. The detection method is as follows: using the method of image segmentation to obtain the "text - non - text" segmentation map and the "blue pixel region - non - blue pixel region" segmentation map of the image; calculating the black text segmentation map and the blue text segmentation map according to the "text - non - text" segmentation map and the "blue pixel region - non - blue pixel region" segmentation map; obtaining the text detection frame according to the connected components of the text regions in each text segmentation map. The advantages of the present invention are: by integrating text color information, the text detection frame is divided into two categories, namely, the field name detection frame and the field value detection frame, effectively solving the problem of adhesion of different types of text caused by printing offset in bill text detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of text recognition, and particularly relates to a bill text detection method that fuses the prior knowledge of text color. Background Art

[0002] In recent years, deep learning technology has developed rapidly and has been applied to various practical scenarios. In the aspect of image recognition, text detection methods based on deep learning are widely adopted in the industry.

[0003] The text content of a bill contains two types of fields: one is the bill content printed later, that is, the field value, and the font color is generally blue; the other is the content on the blank bill without printed content of the field value, that is, the field name, and the font color is generally black. In actual business requirements, it is often necessary to match the detected field values and field names. However, due to the problem of printing offset, the field values and field names on the bill often appear too close or even overlapping, resulting in the problem of adhesion of the detection frames of the field name and the field value, and dividing the field name and the field value into the same detection frame, which brings difficulties to field matching. Summary of the Invention

[0004] The purpose of the present invention is to provide a bill text detection method that fuses the prior knowledge of text color according to the deficiencies of the above-mentioned prior art. The bill text detection method obtains a "text-non-text" segmentation map and a "blue pixel region-non-blue pixel region" segmentation map of the bill image through an image segmentation method, and calculates a black text segmentation map and a blue text segmentation map based on the above two segmentation maps, and obtains a text detection frame according to the connected components of the text regions in each text segmentation map.

[0005] The realization of the purpose of the present invention is completed by the following technical solutions:

[0006] A bill text detection method that fuses the prior knowledge of text color, characterized in that the detection method includes the following steps:

[0007] S1: Scan to obtain a bill image;

[0008] S2: Use an image segmentation method to obtain a "text-non-text" segmentation map of the bill image B t and a "blue pixel region-non-blue pixel region" segmentation map B b ; wherein, the "text-non-text" segmentation map B t refers to dividing the bill image into two parts: a text region and a non-text region, and the "blue pixel region-non-blue pixel region" segmentation map B bIt means dividing the bill image into two parts: the area near blue text pixels and the non - blue pixel area;

[0009] S3: Through a logical operation formula, convert the "text - non - text" segmentation map B t and the "blue pixel area - non - blue pixel area" segmentation map B b into a black text segmentation map B black and a blue text segmentation map B blue ; The logical operation formula is:

[0010] B blue = B t ∧ B b ;

[0011] B black = B t ⊕ B b ;

[0012] Among them, ∧ represents the AND operation; ⊕ represents the XOR operation;

[0013] S4: Calculate the border of each connected component in the black text segmentation map B black to obtain the text detection box of the black field; Calculate the border of each connected component in the blue text segmentation map B blue to obtain the field value detection box.

[0014] The method for obtaining the "text - non - text" segmentation map B t of the bill image in step S2 includes the following steps: Based on the deep learning method, use the pyramid network to extract image features, and based on the image features, use the fully convolutional network to perform semantic segmentation on the bill image, and the network outputs the probability that each pixel point in the bill image is text, so as to obtain the "text - non - text" segmentation map B t .

[0015] The method for obtaining the "blue pixel area - non - blue pixel area" segmentation map B bThe method includes the following steps: Based on deep learning, use an image segmentation network to perform semantic segmentation on the bill image, and divide the pixels of the bill image into two parts: the area near the blue text pixels and the non-blue pixel area, so as to obtain a "blue pixel area - non-blue pixel area" segmentation map. B b 。

[0016] In step S4, obtain a black text segmentation map. B black The method for obtaining the text detection box in is: Use a binary image contour extraction algorithm based on raster scanning to obtain a black text segmentation map. B black Obtain the boundary of the connected domain in, and take the smallest rectangle that can enclose the boundary as the final text detection box.

[0017] In step S4, obtain a blue text segmentation map. B blue The method for obtaining the field value detection box in is: Use a binary image contour extraction algorithm based on raster scanning to obtain a blue text segmentation map. B blue Obtain the boundary of the connected domain in, and take the smallest rectangle that can enclose the boundary as the final field value detection box.

[0018] The advantage of the present invention is that by fusing text color information, the text detection box is divided into two categories: the field name detection box and the field value detection box, effectively solving the problem of adhesion of different categories of text in bill text detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flow chart of the bill text detection method that fuses text color prior in the present invention;

[0020] Figure 2 is a schematic diagram of the bill image in the present invention;

[0021] Figure 3 is a schematic diagram of the "text - non-text" segmentation map in the present invention;

[0022] Figure 4 is a schematic diagram of the "blue pixel area - non-blue pixel area" segmentation in the present invention;

[0023] Figure 5 is a schematic diagram of the blue text segmentation map in the present invention;

[0024] Figure 6 is a schematic diagram of the black text segmentation map in the present invention;

[0025] Figure 7 is a schematic diagram of the blue text detection box in the present invention;

[0026] Figure 8 Schematic diagram of the black text detection frame in the present invention. Specific embodiments

[0027] The features of the present invention and other related features are further described in detail below through embodiments with reference to the accompanying drawings for the understanding of those skilled in the same industry:

[0028] Embodiment: As Figure 1-8 shown, this embodiment specifically relates to a bill text detection method integrating prior knowledge of text color. The bill text detection method includes the following steps:

[0029] S1: As Figure 2 shown, for a bill with the phenomenon of blue text printing offset, use an image scanning device to scan and obtain the bill image and upload it for processing.

[0030] Among them, the solid font represents black text, which is the field name in the bill; the horizontal stripe font represents blue text, which is the field value in the bill. There is a printing offset phenomenon for the blue text, and it is not completely aligned with the table on the bill. Due to the printing offset, the position of a field value "North of L Road" and a field name "Pouring method" in the bill image is close. If the prior knowledge of text color is not used during the text detection process, it is easy to classify the two into the same text detection frame, bringing difficulties to subsequent field matching in actual business.

[0031] S2: Use the method of image segmentation to obtain the "text - non - text" segmentation map of the bill image B t and the "blue pixel area - non - blue pixel area" segmentation map B b ;

[0032] As Figure 3 shown, the "text - non - text" segmentation map B t refers to dividing the bill image into two parts: the text area and the non - text area. In the "text - non - text" segmentation map B t , white represents the position of the text in the bill image of this embodiment. By comparison Figure 2 it can be seen that: in this embodiment, a field value "North of L Road" and a field name "Pouring method" in the bill image are adhered to each other, which is reflected as the segmentation areas of these two fields being combined into the same connected domain in the "text - non - text" binary map B t . If not processed, these two fields will be divided into the same text detection frame. Using the prior information of text color can solve this problem. Obtain the "text - non - text" segmentation map of the bill image Bt The method includes the following steps: Based on deep learning, use a pyramid network to extract image features, and based on the image features, use a fully convolutional network to perform semantic segmentation on the bill image. The network outputs the probability that each pixel point in the bill image is text to obtain a "text - non - text" segmentation map. B t 。

[0033] As Figure 4 shown, the "blue pixel area - non - blue pixel area" segmentation map B b refers to dividing the bill image into two parts: the area near the blue text pixels and the non - blue pixel area, that is, separating the area except for the area near the blue pixel points; obtaining the "blue pixel area - non - blue pixel area" segmentation map of the bill image. B b The method includes the following steps: Based on deep learning, use an image segmentation network to perform semantic segmentation on the bill image, and divide the pixels of the bill image into two parts: the area near the blue text pixels and the non - blue pixel area to obtain the "blue pixel area - non - blue pixel area" segmentation map. B b 。

[0034] S3: Through a logical operation formula, convert the "text - non - text" segmentation map obtained in step S2 B t and the "blue pixel area - non - blue pixel area" segmentation map B b into the blue text segmentation map as Figure 5 shown and the black text segmentation map as B blue and as Figure 6 shown; B black ;

[0035] Among them, the logical operation formula is:

[0036] B blue = B t ∧ B b ;

[0037] B black = B t ⊕ B b ;

[0038] Among them, ∧ represents the AND operation; ⊕ represents the XOR operation.

[0039] S4: Calculate the black text segmentation map B black For each connected component in B blue calculate the bounding box to obtain the text detection box for the black fields; calculate the blue text segmentation map

[0040] As Figure 7 shown, the method to obtain the text detection box for the black text segmentation map B black is as follows: Use the binary image contour extraction algorithm based on raster scanning to obtain the black text segmentation map B black and the boundaries of the connected components in it, and take the smallest rectangle that can enclose the boundaries as the final text detection box, thus obtaining the text (field name) detection box in the bill image of this embodiment. The text contents of the field name detection boxes detected in this embodiment are respectively: "Concrete Delivery Visa Form of Company A", "Project Name", "Date", "Material Issuance Time", "Construction Location", "Pouring Method", "Volume of This Truck", "Cumulative Volume".

[0041] As Figure 8 shown, the method to obtain the field value detection box for the blue text segmentation map B blue is as follows: Use the binary image contour extraction algorithm based on raster scanning to obtain the blue text segmentation map B blue and the boundaries of the connected components in it, and take the smallest rectangle that can enclose the said boundaries as the final said field value detection box. The text contents of the detected field value detection boxes are respectively: "Rapid Reconstruction Project of Road L in City K", "2021 / 08 / 28", "14:56", "North of Road L", "Non - Pump", "3.20", "15.60".

[0042] In this embodiment, the present invention effectively solves the problem of the adhesion of detection boxes of different types of texts caused by printing offset in bill text detection by integrating the prior knowledge of text color during the detection process.

Claims

1. A bill text detection method integrating prior knowledge of text color, characterized in that The detection method includes the following steps: S1: Scan to obtain a bill image; S2: Use the method of image segmentation to obtain the "text-non-text" segmentation map of the bill image B t and the "blue pixel area-non-blue pixel area" segmentation map B b ; Among them, the "text-non-text" segmentation map B t refers to dividing the bill image into two parts: a text area and a non-text area, and the "blue pixel area-non-blue pixel area" segmentation map B b refers to dividing the bill image into two parts: the area near the blue text pixels and the non-blue pixel area; Obtain the "text-non-text" segmentation map of the bill image B t The method includes the following steps: Based on deep learning, use a pyramid network to extract image features, and based on the image features, use a fully convolutional network to perform semantic segmentation on the bill image. The network outputs the probability that each pixel point in the bill image is text, so as to obtain the "text-non-text" segmentation map B t ; Obtain the "blue pixel area - non - blue pixel area" segmentation map of the bill image B b The method includes the following steps: Based on deep learning, use an image segmentation network to perform semantic segmentation on the bill image, divide the pixels of the bill image into two parts: the area near the blue text pixels and the non - blue pixel area, so as to obtain the "blue pixel area - non - blue pixel area" segmentation map B b ; S3: Divide the "text - non - text" segmentation map B t and the "blue pixel area - non - blue pixel area" segmentation map B b into a black text segmentation map B black and a blue text segmentation map B blue ; The logical operation formula is: B blue = B t ∧ B b ； B black = B t ⊕ B b ； Wherein, ∧ represents an AND operation; ⊕ represents an XOR operation; S4: Calculate the black text segmentation map B black For each connected component in, obtain the text detection box of the black field; Calculate the blue text segmentation map B blue For each connected component in, obtain the field value detection box; Obtain the black text segmentation map B black The method of the text detection box described in B is as follows: Use a binary image contour extraction algorithm based on raster scanning to obtain the black text segmentation map B black The boundary of the connected domain in B is used to obtain the smallest rectangle that can enclose the boundary as the final text detection box Obtain the blue text segmentation map B blue The method for the field value detection box described in B is: use a binary image contour extraction algorithm based on raster scanning to obtain the blue text segmentation map B blue The boundary of the connected domain in B is used to obtain the smallest rectangle that can enclose the boundary as the final field value detection box

Images