A circular seal recognition method, device and medium based on PSENet

Abstract

The application provides a circular seal recognition method and device based on PSENet and a medium. The application belongs to the technical field of artificial intelligence. The implementation steps are as follows: first, red seals in an image are extracted through an HSV color space; then, a seal region is located by calculating a connected domain, and a seal frame is removed; then, the position of text in the seal is located through PSENet; then, the text is rotated and corrected; then, the corrected text is subjected to OCR recognition; finally, text layout analysis is performed. In the HSV color space, the independence between channels is stronger, and the color characteristics of the seal can be better reflected, so that the circular seal in the document image can be effectively extracted; the scheme of correcting the direction first and then performing OCR recognition on the text in the seal can effectively improve the recognition accuracy of the seal text.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a circular seal recognition method, device and medium based on PSENet. BACKGROUND

[0002] Seals are the highest legal certification, and are important certification tools used on official announcements, contract agreements, license documents and the like, containing important information of the issuing authority. Seal recognition is an important link in document image recognition, and accurate extraction and recognition of seals are of great significance to document image recognition analysis. Seals are generally the last link in the generation of documents and are added to the blank area of the document or cover the text area, and the latter case is more common. Therefore, on the one hand, how to accurately separate the seal from the document image, especially the image in which the seal and the text overlap, is a difficulty in document image processing. On the other hand, most seals are circular, and the text therein is in a circular layout, which also brings certain difficulty to seal text recognition.

[0003] A common seal extraction method is a method based on image color features, i.e. using the color features of the seal. One implementation is to analyze the difference between the red channel and the blue and green channels in the RGB color space of the image to extract red pixels constituting the seal, but this method is more dependent on the selection of the difference threshold. In terms of seal text recognition, a common method is to detect feature points of the extracted seal and match them with existing seal templates, or to transform the text in the seal to the polar coordinate space and then perform OCR recognition. The disadvantage is that the feature matching method has a small scope of application and can only recognize seals with existing templates, while the transformation to the polar coordinate space distorts the text, which is not conducive to text OCR recognition. SUMMARY

[0004] To solve the above problems, the application provides a circular seal recognition method, device and system based on PSENet. First, the red seal in the image is extracted through the HSV color space; then the seal area is located by calculating the connected domain and the seal frame is removed; then the position of the text in the seal is located by PSENet; then the text is rotated and corrected; then the corrected text is subjected to OCR recognition; and finally, text layout analysis is performed.

[0005] The specific steps of the method are as follows:

[0006] Step 1: Seal extraction: red seal pixels in the document image are extracted through the HSV color space feature.

[0007] Step 2: Seal positioning and frame removal: the seal area is located by calculating the connected domain range of the red pixel area, and the seal frame is removed.

[0008] Step three, text position positioning: for the extracted seal image, the boundary box information of the text in the seal region is positioned through the PSENet network.

[0009] Step four, text rotation correction: according to the detected text frame information in step three, the rotation angle of each text block is calculated, and the direction of the text is corrected by reverse rotation.

[0010] Step five, OCR recognition: the text block result corrected in step four is subjected to OCR recognition, and the text content is recognized.

[0011] Step six, text layout analysis: according to the position information obtained in step three, the text block is analyzed and divided into two categories: the issuing authority content of the circular layout, and the seal name content of the horizontal layout, and the text block recognition result of step five is combined.

[0012] Further, step one is specifically: converting the image in the original RGB color space into a representation using the HSV color space, analyzing the range of HSV components to extract red pixel regions; wherein the range of H channel is 0-10 or 156-180, the range of S channel is 43-255, and the range of V channel is 46-255.

[0013] Further, step two is specifically: the connected domain is the largest region composed of mutually connected pixel points in the image, and the connected domain with the largest boundary range in the extracted red region is the region where the seal frame is located, so as to locate the seal region and obtain the center of the seal, the coordinate information of the seal on the image is represented as (x, y), the frame is removed, and only the text in the seal is retained.

[0014] Further, step three is specifically: the text block boundary box information positioned by PSENet is represented as (x1, y1, x2, y2, x3, y3, x4, y4), wherein (x1, y1), (x2, y2), (x3, y3) and (x4, y4) are respectively the coordinates of the left upper corner point, the right upper corner point, the right lower corner point and the left lower corner point of the text block boundary box; the center position coordinates of the text block are calculated and represented as (x5, y5), wherein:

[0015]

[0016]

[0017] Further, the calculation method of the rotation angle of the text block is:

[0018] (1) The calculation formula of the rotation angle of each text block is:

[0019]

[0020] Among them, y1 represents the ordinate of the upper left corner point of the text block bounding box; y4 represents the ordinate of the lower left corner point of the text block bounding box; x1 represents the abscissa of the upper left corner point of the text block bounding box; x4 represents the abscissa of the lower left corner point of the text block bounding box; angle represents the rotation angle of the text block.

[0021] (2) In addition, it is also necessary to perform a secondary judgment on the calculated rotation angle by combining the center position coordinates (x5, y5) of the text block and the center coordinates (x, y) of the seal:

[0022]

[0023] Further, Step 6 is specifically as follows: Combine the position information (x, y) of the seal center obtained in Step 2 and the position information (x5, y5) of the centers of all text blocks obtained in Step 3, and calculate the distance between the centers of all text blocks and the seal center in the x direction, that is, |x5 - x|; for text blocks with a distance less than the set threshold, if the y5 of the text block > y, it is the seal name, and if the y5 of the text block < y, it is the text block at the center position of the issuing authority content, where the threshold is set to the minimum width among all text block bounding boxes; for text blocks with a distance greater than the set threshold, they are other text blocks of the issuing authority content; combine the center coordinates (x5, y5) of the text blocks with the determined center positions, and sort all the text blocks belonging to the issuing authority content in ascending order according to x5, so as to determine the text order of the issuing authority content, and combine the text recognition results according to this order to obtain the final recognition result.

[0024] The present invention provides an electronic device, including a memory and a processor, the memory stores a computer program, and it is characterized in that: when the processor executes the computer program, the steps of any one of the circular seal recognition methods based on PSENet can be realized.

[0025] The present invention provides a computer-readable storage medium for storing computer instructions, and it is characterized in that: when the computer instructions are executed by a processor, the steps of any one of the circular seal recognition methods based on PSENet can be realized.

[0026] The beneficial effects of the present invention are as follows: The present invention proposes a circular seal recognition method based on PSENet. First, the seal is extracted through the processing of the HSV color space, then the seal text is positioned by PSENet, and after each text block is corrected, it is recognized by the OCR model. In the HSV color space, the independence between channels is stronger, which can better reflect the color characteristics of the seal, so the circular seal in the document image can be effectively extracted; the scheme of correcting the direction of the text in the seal first and then performing OCR recognition can effectively improve the accuracy of seal text recognition. Description of the Drawings

[0027] Figure 1 is a scheme flowchart;

[0028] Figure 2 is a PSENet text positioning process;

[0029] Figure 3 is a connected domain in the detection result;

[0030] Figure 4 is a text block correction result. DETAILED DESCRIPTION

[0031] The technical solutions in the present application will be described clearly and completely below in conjunction with embodiments and drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0032] Figure 1 is a scheme flowchart of the present application, and the flowchart of the present application includes:

[0033] Step S1, seal extraction: extracting red seal pixels in a document image through HSV color space features.

[0034] Step S2, seal positioning and frame removal: positioning a seal region through calculation of a connected domain range of a red pixel region, and removing a seal frame.

[0035] Step S3, text position positioning: for the extracted seal image, positioning a text bounding box of the seal region through a PSENet network.

[0036] Step S4, text rotation correction: calculating a rotation angle of each text block according to the detected text bounding box information of step S3, performing reverse rotation, and correcting the direction of the text.

[0037] Step S5, OCR recognition: performing OCR recognition on the text block result corrected in step S4.

[0038] Step S6, text layout analysis: performing layout analysis on the text block according to the position information obtained in step S3, and dividing into two categories: a circular layout of a certificate-issuing authority content and a horizontal layout of a seal name content, and then combining the text block recognition result of step S5.

[0039] In step S1, the HSV color space features are defined as follows: the original RGB color space image is converted to a representation using the HSV color space, and the range of the HSV components is analyzed to extract the red region. Specifically, the range of the H channel is 0–10 and 156–180, the range of the S channel is 43–255, and the range of the V channel is 46–255.

[0040] In step S2, a connected region is the largest region formed by interconnected pixels in an image. Within the extracted red region, the connected region with the largest boundary is the area containing the seal's border, thus locating the seal area and obtaining its center, represented by coordinates (x, y) on the image. To improve the accuracy of subsequent recognition, the border is removed, retaining only the text within the seal.

[0041] In step S3, PSENet can locate curved text, thus it is suitable for detecting and locating circular text within a seal. The text localization process based on PSENet is shown in the attached figure. Figure 2 As shown, it mainly consists of two parts: the feature pyramid network on the left is used for image feature extraction, and the feature fusion part on the right is used to implement a progressive expansion algorithm for the segmentation results.

[0042] The feature pyramid network generates feature maps at four different scales. Figures 1 to 4 Each feature map has a dimension of 256. Module C processes the data by upsampling the four feature maps at different scales (feature maps...). Figure 2 Perform 2x upsampling, features Figure 3 Perform 4x upsampling, features Figure 4 (8x upsampling) uniformly adjusted to feature Figure 1 The dimensions are then spliced ​​together, resulting in 1024 dimensions.

[0043] The processing in module F is as follows: First, the dimension of the feature map is reduced to 256 through a convolutional layer. Then, through a series of convolutions, upsampling, and sigmoid calculations, n segmentation results are obtained. After that, starting from segmentation result 1, a progressive expansion algorithm is used to fuse the segments layer by layer upwards.

[0044] Specifically, the progressive expansion algorithm works as follows: starting with segmentation result 1, it searches for connected components in the image based on the segmentation results and assigns different labels, as shown in the appendix. Figure 3 As shown; then, combining the next segmentation result, a breadth-first search is performed. If the current pixel position is (x0, y0) and the label is 1, for its neighboring pixels, if they have no label and are considered to be in the text category in the segmentation result, then label 1 is added.

[0045] The information of the character block boundary box positioned by the PSENet in the step S4 is represented as (x1, y1, x2, y2, x3, y3, x4, y4), wherein (x1, y1), (x2, y2), (x3, y3), (x4, y4) are respectively the coordinates of the top-left corner point, the top-right corner point, the bottom-right corner point, and the bottom-left corner point of the character block boundary box.

[0046] The center position coordinates of the character block are calculated and represented as (x5, y5), wherein

[0047]

[0048]

[0049] The calculation of the tilt angle of each character block is as follows:

[0050]

[0051] wherein y1 represents the vertical coordinate of the top-left corner point of the character block boundary box; y4 represents the vertical coordinate of the bottom-left corner point of the character block boundary box; x1 represents the horizontal coordinate of the top-left corner point of the character block boundary box; x4 represents the horizontal coordinate of the bottom-left corner point of the character block boundary box; and angle represents the rotation angle of the character block.

[0052] In addition, the tilt angle calculated needs to be judged again in combination with the center position coordinates (x5, y5) of the character block and the center coordinates (x, y) of the seal:

[0053]

[0054] wherein angle represents the tilt angle of the character block.

[0055] Finally, the reverse tilt correction is performed according to the tilt angle of each character block, and the correction result of the character block is as shown in Figure 4

[0056] In the step S6, the layout analysis of the character block is performed.

[0057] In combination with the position information (x, y) of the center of the seal obtained in the step S2 and the position information (x5, y5) of the center of all the character blocks obtained in the step S4, the distance between the center of each character block and the center of the seal in the x direction, i.e., |x5-x|, is calculated.

[0058] For the character block with a distance less than a set threshold value, if y5>y, it is the character block of the name of the seal, and if y5

[0059] ​For the text blocks with distance greater than the set threshold, i.e. other text blocks of the content of the issuing authority, the text blocks belonging to the content of the issuing authority are sorted in ascending order of x5 according to the center coordinates (x5, y5) of the text blocks with the determined center positions, so as to determine the text order of the content of the issuing authority. The text recognition result is combined according to the order, and the final recognition result is obtained.

Claims

1. A circular stamp recognition method based on PSENet, characterized in that: The steps of the identification method include: Step 1, Seal Extraction: Extract the red seal pixels from the document image using HSV color space features; Step 2, Seal Location and Border Removal: The seal area is located by calculating the connected component range of the red pixel area, and the seal border is removed. The connected component is the largest area formed by interconnected pixels in the image. The connected component with the largest boundary range in the extracted red area is the area where the seal border is located, thus locating the seal area and obtaining the center of the seal. The coordinate information of the seal on the image is represented as (x, y). The border is removed, and only the text inside the seal is kept. Step 3, text location: For the extracted seal image, the bounding box information of the text in the seal area is located using the PSENet network; Step 4, Text Rotation Correction: Based on the text border information detected in Step 3, calculate the rotation angle of each text block, perform reverse rotation, and correct the text orientation. Step 5, OCR recognition: Perform OCR recognition on the text block results corrected in Step 4 to recognize the text content; Step Six, Text Layout Analysis: Based on the position information obtained in Step Three, the text blocks are analyzed for layout, divided into two categories: circular layout of the issuing authority content and horizontal layout of the seal name content, and then combined with the text block recognition results from Step Five.

2. The circular stamp recognition method based on PSENet according to claim 1, characterized in that: Step one specifically involves converting the original RGB color space image into a representation using the HSV color space, analyzing the range of the HSV components to extract the red pixel region; wherein, the range of the H channel is 0 to 10 or 156 to 180, the range of the S channel is 43 to 255, and the range of the V channel is 46 to 255.

3. The circular stamp recognition method based on PSENet according to claim 1, characterized in that: Step three specifically involves: The bounding box information of the text block within the stamp located using PSENet is represented as (x1, y1, x2, y2, x3, y3, x4, y4), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the coordinates of the top-left, top-right, bottom-right, and bottom-left corners of the text block's bounding box, respectively; the center position coordinates of the text block are calculated, represented as (x5, y5), where: ； 。 4. The circular stamp recognition method based on PSENet according to claim 1, characterized in that: The method for calculating the rotation angle of the text block is as follows: (1) The formula for calculating the rotation angle of each text block is: Where y1 represents the ordinate of the top left corner of the text block bounding box; y4 represents the ordinate of the bottom left corner of the text block bounding box; x1 represents the abscissa of the top left corner of the text block bounding box; x4 represents the abscissa of the bottom left corner of the text block bounding box; and angle represents the rotation angle of the text block. (2) In addition, the calculated rotation angle needs to be judged a second time by combining the center position coordinates (x5, y5) of the text block and the center coordinates (x, y) of the seal: 。 5. The circular stamp recognition method based on PSENet according to claim 1, characterized in that: Step 6 is specifically as follows: Combining the position information (x, y) of the seal center obtained in Step 2 and the position information (x5, y5) of the centers of all text blocks obtained in Step 3, calculate the distance in the x direction between the centers of all text blocks and the seal center, that is, |x5 - x|; for text blocks with a distance less than the set threshold, if y5 > y of the text block, it is the seal name, and if y5 < y of the text block, it is the text block at the center position of the issuing authority content, where the threshold is set to the minimum width among the bounding boxes of all text blocks; for text blocks with a distance greater than the set threshold, they are other text blocks of the issuing authority content; combining the center coordinates (x5, y5) of the text blocks with the determined center positions, sort all the text blocks belonging to the issuing authority content in ascending order of x5, so as to determine the text order of the issuing authority content, and combine the text recognition results according to this order to obtain the final recognition result.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-5.

7. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by the processor, it implements the steps of the method according to any one of claims 1-5.

Images