A method, system and medium for recognizing seal content
By employing a streamlined seal detection and recognition method, combined with deep neural networks and arc curve correction algorithms, the accuracy problem of seal content recognition in complex backgrounds is solved, achieving efficient seal content extraction and recognition, adapting to various scenarios, and saving labor costs.
Patent Information
- Application Number
- CN202311723322.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-11
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-06-11
AI Technical Summary
Existing technologies do not achieve high accuracy in recognizing stamp content in complex backgrounds, especially for curved text. Furthermore, existing methods suffer from issues such as mismatched curve fitting, text distortion, or segmentation after correction.
It adopts a pipeline-style method for seal detection, content extraction and recognition, combined with a seal content region expansion algorithm based on a deep neural network model, and uses a correction algorithm based on arc curves to correct curved text. It also combines horizontal text detection and arbitrary shape text detection to adapt to different seal categories.
It achieves high-precision seal content detection and recognition with limited real data, adapts to various scenarios, saves labor costs, and improves office efficiency.
Smart Images

Figure CN118334670B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision target detection and text recognition technology, and in particular to a method, system and medium for recognizing seal content based on arc curve correction. Background Technology
[0002] With advancements in science and technology, computer-trained algorithms can scan, read, and understand digital and paper documents like humans, giving rise to the concept of intelligent document processing (IDP). IDP refers to the automatic processing of documents by computers based on their understanding. Common forms include automatic or semi-automatic document generation, automatic document understanding, integration of documents with business processes, automatic document evaluation, and document auditing. This technology is becoming increasingly popular in many fields. IDP can help enterprises automate their daily document processing tasks, providing assistance to employees in various aspects such as document recognition, classification, information extraction, and comparison.
[0003] Seal recognition is a crucial capability in intelligent document processing. With the advancement of image processing technology, technologies such as Optical Character Recognition (OCR) have also made significant progress. Along with the widespread adoption of paperless and electronic office practices, the digitalization of seals has become a trend. Seals, as legally significant symbols and evidence, are important tokens used by government agencies at all levels, social organizations, and enterprises, playing a vital role in modern life. They are widely used in contract comparison, data entry systems, inventory management, and invoice reimbursement. Previously, these processes required manual verification of seal images, a cumbersome procedure. Using artificial intelligence technology for automatic seal detection and recognition can efficiently extract the text content from the seal, solving timeliness and accuracy issues in tasks such as contract management. Simultaneously, it can significantly reduce labor costs for enterprises, demonstrating immense practical value.
[0004] Compared to text recognition in ordinary scenarios, seal recognition has some unique characteristics, which are also its main challenges. First, seals are highly diverse. Based on type, there are official seals, contract seals, financial seals, and legal representative seals, among others. Based on shape, there are square, rhomboid, circular, and oval seals, etc. The text in square seals is horizontally distributed, and current methods for detecting and recognizing horizontal text are quite mature. After rotating the square seal to the horizontal orientation, common OCR recognition models can easily identify the content. However, the text in common seals is arranged in a circular pattern, and recognizing this type of curved text still faces significant challenges. Second, seals are often accompanied by complex backgrounds. In real-world scenarios, seals are often affixed to various documents, and the background of the seal area is complex, obscured and interfered with by factors such as the text on the document itself, handwritten signatures, etc. Furthermore, factors such as the quality of the ink and the pressure applied during stamping can affect the seal image, easily resulting in uneven color, blurred text, and even difficulty in recognition by the human eye. Therefore, there is an urgent need to propose a method for recognizing stamp content with complex backgrounds in various documents in order to solve this practical problem.
[0005] Existing methods for recognizing stamp image content have certain problems. On the one hand, in complex document backgrounds, accurate stamp content areas are more difficult to detect, requiring higher accuracy from text detection algorithms. Existing text detection algorithms do not perform ideally in such scenarios. On the other hand, methods for recognizing curved stamp content areas need to be adapted to the characteristics of the stamp. Existing stamp image content recognition methods based on Bézier curve correction use Bézier curves to fit the boundaries of the stamp content areas. While this curve can fit any curve in two-dimensional space, because it is fitted from a point set using computational methods, the fitted curve may not perfectly match the actual text boundaries, leading to issues such as the stamp content being cut by the curve and excessive distortion of the text after correction. Additionally, there are stamp image content recognition methods based on polar coordinate transformation. This method requires extensive post-processing for acquiring and correcting the stamp content areas, and suffers from the problem of segmented continuous text areas after correction, requiring further post-processing to re-merge them. Other stamp recognition methods based on image feature matching do not truly recognize the stamp content and have very poor general applicability. Summary of the Invention
[0006] The main objective of this invention is to propose a method, system, and medium for seal content recognition, aiming to solve the problems of seal detection, classification, and recognition in various scenarios. In the seal text recognition part, a pipeline process from text detection to correction and then to recognition is adopted. Compared with end-to-end recognition methods, this invention has better adaptability to different seal detection and recognition application scenarios. In particular, a seal content region expansion algorithm based on a deep neural network model is added to the text detection part, which can obtain more accurate seal content region boundaries. The entire framework can achieve good results even with a relatively small amount of training data.
[0007] To achieve the above objectives, the present invention proposes a method for recognizing the content of a seal, the method comprising the following steps:
[0008] Seal position detection steps: acquire an image with a seal, execute a seal detection and classification algorithm to detect and locate the seal in the image, and obtain the position coordinates and category of the seal in the image. The seal categories include circle, ellipse, rhombus and square.
[0009] Seal content detection steps: The image area of the seal is cropped according to the position coordinates of the seal, and the seal content detection algorithm is executed on the image area of the seal to obtain the seal content area represented by a list of polygon coordinates;
[0010] Seal content area correction steps: Sort the polygon coordinates, and for seal content areas of the seal category of circle and ellipse, execute the seal content area correction algorithm based on arc curve to correct the circle or ellipse seal content area into a rectangle to obtain the corrected seal content area.
[0011] Seal content recognition steps: For the content area of a rhombus or square seal, or the corrected content area of the seal, execute the seal content recognition algorithm to output the text content of the seal.
[0012] A further technical solution of the present invention is that the seal position detection step includes:
[0013] Obtain the feature map of the input image and construct multiple candidate boxes that may contain the seal;
[0014] The candidate boxes are subjected to binary classification and regression, and the filtered portion is determined to be the background.
[0015] Perform candidate box-feature map alignment on the remaining candidate boxes to obtain the feature representation of the corresponding region on the image feature map. Then, perform multi-classification and border position offset regression on the remaining candidate boxes to obtain the stamp position coordinates and category.
[0016] A further technical solution of the present invention is that the seal content detection step includes:
[0017] Using all the seal images and corresponding category information obtained in the seal detection step as input information for the seal content detection step, content detection is performed on each seal according to the corresponding detection method based on the category.
[0018] In the case where the seal is square or rhomboid, a horizontal text detection method is used to perform a horizontal text detection algorithm on the seal image to obtain the seal content area.
[0019] When the seal category is circular or elliptical, an arbitrary shape text detection method is used to perform an arbitrary shape text detection algorithm on the seal image to obtain the seal content area.
[0020] A further technical solution of the present invention is that, when the seal type is circular or elliptical, the step of using an arbitrary shape text detection method to perform an arbitrary shape text detection algorithm on the seal image to obtain the seal content area includes:
[0021] For seal images classified as circular or elliptical, a text detection algorithm based on instance segmentation is executed to obtain a pixel classification probability map and a seal image feature map. A relatively large probability threshold is selected to ensure that the initial seal content regions are all inside the real seal content regions. A binarization method is performed on the pixel classification probability map to obtain a binarized image. A seal content region bounding box generation algorithm is then performed on the binarized image to obtain multiple initial bounding box coordinate sequences for the seal content regions.
[0022] Using multiple initial bounding box coordinate sequences of the seal content region, feature vectors corresponding to each coordinate are extracted from the seal image feature map to obtain feature representations corresponding to multiple text instances. The feature representations corresponding to multiple text instances are then input into a seal content region expansion algorithm based on a deep neural network model to obtain the predicted displacement distance and displacement direction of each coordinate point. The displacement distance and displacement direction of each coordinate point are then applied to the corresponding coordinate point to obtain a more accurate bounding box coordinate sequence of the seal content region compared to directly binarizing based on a preset threshold.
[0023] A further technical solution of the present invention is that the stamp content area correction step includes:
[0024] If the stamp content area obtained by the horizontal document detection method is determined to be rectangular, a rectangular rotation and leveling algorithm is executed to correct the stamp content area to the horizontal direction, thereby obtaining a horizontal stamp content image.
[0025] For the coordinate sequence of the stamp content area returned by the arbitrary shape text detection method, if it is determined that the coordinate sequence of the stamp content area forms a polygon to be classified, the polygon shape judgment method is executed on the stamp content area. The area of the area is obtained through the coordinate sequence of the stamp content area. At the same time, the minimum generated rectangle including this stamp content area is generated, and the area of the minimum generated rectangle is calculated. The ratio of the area of the stamp content area to the area of the corresponding minimum generated rectangle is calculated and compared with a set threshold. If it is greater than the threshold, the shape of the stamp content area is determined to be a rectangle; if it is less than the threshold, the shape of the stamp content area is determined to be an arc.
[0026] For the rectangular stamp content area, perform a rectangular rotation and leveling algorithm to adjust it to the horizontal direction, resulting in a horizontal stamp content image;
[0027] For the arc-shaped seal content area, a seal content area correction algorithm based on the arc curve is executed. The special characteristic of the curved text area of the seal is that it is straightened and corrected to obtain a horizontal seal content image.
[0028] A further technical solution of the present invention is that the stamp content area correction step further includes:
[0029] The post-processing method is executed to rotate the seal image to the positive direction according to the coordinate sequence of the seal content area. At the same time, the coordinate sequence is rearranged and appropriately expanded to obtain the coordinate sequence of the seal content area, which rotates clockwise from the upper left corner of the semantic seal content.
[0030] The inner and outer arc coordinate sequence calculation algorithm is executed on the coordinate sequence of the seal content area to obtain two coordinate sequences representing the inner and outer arcs respectively. Then, the circle fitting calculation method is executed to obtain the center coordinates and radius of the inner and outer arcs, as well as the equation of the circle fitted by the inner and outer arcs.
[0031] The width and height of the output image are preset. Pixels are selected in the output image in row and column order. The proportion of the horizontal and vertical positions of the pixels is calculated. Using the proportion, based on the fitted equation of the circle and the coordinates of the start and end points of the inner and outer arcs, a point matching the pixel in the output image is determined in the seal content area. Using an interpolation algorithm, the pixel value corresponding to the pixel in the output image is calculated based on the pixel value information of the seal content area.
[0032] Calculate the pixel value of each pixel in the output image to obtain the horizontal stamp content image.
[0033] A further technical solution of the present invention is that the seal content recognition step includes:
[0034] A character recognition algorithm is performed on the horizontal seal content image to obtain the text content in the seal content image.
[0035] To achieve the above objectives, the present invention also proposes a seal content recognition system, the system comprising a memory, a processor, and a seal content recognition program stored on the processor, wherein the seal content recognition program is executed by the processor to perform the steps of the method described above.
[0036] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a seal content recognition program, wherein the seal content recognition program is executed by a processor to perform the steps described in the above embodiments.
[0037] Compared with existing technologies, the advantages of this invention are: the streamlined process of seal detection, content extraction, correction, and recognition reduces the need for real seal annotation data while still achieving high accuracy in detection, classification, content extraction, and recognition. Furthermore, considering the complex backgrounds and noise interference often present in real-world scenarios, the generated data used in model training is designed to closely resemble reality, ensuring the adaptability and effectiveness of this invention across various scenarios. This invention effectively solves the problem of seal content recognition in scenarios such as contract entry systems, inventory management, and invoice reimbursement, comparing and verifying the names of contract or invoice entities with the seal content, saving significant labor costs and improving enterprise office efficiency. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0039] Figure 1 This is a flowchart of a preferred embodiment of the seal content recognition method in this invention.
[0040] Figure 2 This is the present invention. Figure 1 A flowchart of a preferred embodiment of step S200.
[0041] Figure 3 This is the present invention. Figure 1 Flowchart for stamp content detection when a square or rhombus-shaped stamp is input in step S300.
[0042] Figure 4 This is the present invention. Figure 1Flowchart for stamp content detection when inputting a circular or oval stamp in step S300.
[0043] Figure 5 This is the present invention. Figure 1 In step S400, the flowchart shows the correction process when the content area of the seal is determined to be rectangular.
[0044] Figure 6 This is the present invention. Figure 1 In step S400, the flowchart for the correction when the content area of the seal is determined to be an arc shape is shown.
[0045] Figure 7 This is a real-world example illustration.
[0046] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0048] This invention proposes a method for recognizing the content of a seal, such as... Figure 1 As shown, a preferred embodiment of the seal content recognition method of the present invention includes the following steps:
[0049] Seal location detection steps: Obtain an image containing a seal, execute a seal detection and classification algorithm to detect and locate the seal in the image, and obtain the seal's position coordinates and category in the image. The seal category includes, but is not limited to, circles, ovals, rhombuses, or squares.
[0050] Seal content detection steps: The image area of the seal is cropped according to the position coordinates of the seal, and the seal content detection algorithm is executed on the image area of the seal to obtain the seal content area represented by a list of polygon coordinates.
[0051] Seal content area correction steps: Sort the polygon coordinates, and for seal content areas of circular and elliptical seal categories, execute a seal content area correction algorithm based on arc curves to correct the circular or elliptical seal content areas into rectangles, thus obtaining the corrected seal content areas.
[0052] Seal content recognition steps: For the content area of a rhombus or square seal, or the corrected content area of the seal, execute the seal content recognition algorithm to output the text content of the seal.
[0053] Specifically, the steps for detecting the position of the seal include:
[0054] Obtain the feature map of the input image and construct multiple candidate boxes that may contain the seal.
[0055] Binary classification and regression are performed on the candidate boxes, and the filtered candidate boxes are identified as background.
[0056] Perform candidate box-feature map alignment on the remaining candidate boxes to obtain the feature representation of the corresponding region on the image feature map. Then, perform multi-class classification and border position offset regression on the remaining candidate boxes to obtain the stamp position coordinates and category.
[0057] The steps for checking the content of a seal include:
[0058] Using all the seal images and corresponding category information obtained in the seal detection step as input information for the seal content detection step, content detection is performed on each seal according to the corresponding detection method based on its category.
[0059] In cases where the seal is square or rhomboid, a horizontal text detection method is used. The horizontal text detection algorithm is applied to the seal image, which is then input into the horizontal text detection model to obtain the seal content region. This horizontal text detection model is trained using sample data from text detection in ordinary documents.
[0060] When the seal type is circular or elliptical, an arbitrary shape text detection method is used. This method applies the arbitrary shape text detection algorithm to the seal image, inputting the image into the arbitrary shape text detection model to obtain the seal content region. The arbitrary shape text detection model is pre-trained using a large number of generated seal content detection annotation samples, and fine-tuned using a small amount of content detection annotation data from various real seals.
[0061] When the seal type is circular or elliptical, the steps for performing arbitrary shape text detection on the seal image to obtain the seal content area include:
[0062] For seal images classified as circular or elliptical, a text detection algorithm based on instance segmentation is executed to obtain a pixel classification probability map and a seal image feature map. A relatively large probability threshold is selected to ensure that the initial seal content regions are all inside the real seal content regions. A binarization method is performed on the pixel classification probability map to obtain a binarized image. A seal content region bounding box generation algorithm is then performed on the binarized image to obtain multiple initial bounding box coordinate sequences for the seal content regions.
[0063] Using multiple initial bounding box coordinate sequences of the seal content region, feature vectors corresponding to each coordinate are extracted from the seal image feature map to obtain feature representations corresponding to multiple text instances. The feature representations corresponding to multiple text instances are then input into a seal content region expansion algorithm based on a deep neural network model to obtain the predicted displacement distance and displacement direction of each coordinate point. The displacement distance and displacement direction of each coordinate point are then applied to the corresponding coordinate point to obtain a more accurate bounding box coordinate sequence of the seal content region compared to directly binarizing based on a preset threshold.
[0064] The steps for correcting the content area of a seal include:
[0065] If the stamp content area obtained by the horizontal document detection method is determined to be rectangular, a rectangular rotation and leveling algorithm is executed to correct the stamp content area to the horizontal direction, thus obtaining a horizontal stamp content image.
[0066] For the coordinate sequence of the stamp content area returned by the arbitrary shape text detection method, if it is determined that the coordinate sequence of the stamp content area forms a polygon to be classified, the polygon shape judgment method is performed on the stamp content area. The area of the area is obtained through the coordinate sequence of the stamp content area. At the same time, the minimum generated rectangle including this stamp content area is generated, and the area of the minimum generated rectangle is calculated. The ratio of the area of the stamp content area to the area of the corresponding minimum generated rectangle is calculated and compared with a set threshold. If it is greater than the threshold, the shape of the stamp content area is determined to be rectangular; if it is less than the threshold, the shape of the stamp content area is determined to be arc-shaped.
[0067] For the rectangular stamp content area, perform a rectangular rotation and leveling algorithm to adjust it to the horizontal direction, resulting in a horizontal stamp content image.
[0068] For the arc-shaped seal content area, a seal content area correction algorithm based on the arc curve is executed. The special characteristic of the curved text area of the seal is that it is straightened and corrected to obtain a horizontal seal content image.
[0069] After classifying the above-mentioned stamp content areas, two types of stamp content areas are obtained: rectangular and arc-shaped. The rectangular stamp content areas are corrected to the horizontal direction using the above-mentioned rectangular rotation and leveling method. For the arc-shaped text boxes, the post-processing method is performed to straighten the stamp image, reorder the coordinates, and slightly expand the stamp content area to improve the correction effect.
[0070] The steps for correcting the content area of the seal also include:
[0071] The post-processing method rotates the seal image to the positive direction based on the coordinate sequence of the seal content area. At the same time, the coordinate sequence is rearranged and appropriately expanded to obtain the coordinate sequence of the seal content area, which rotates clockwise from the upper left corner of the semantic seal content.
[0072] Based on the reordered coordinate sequence, the inner and outer arc coordinate sequence calculation method is applied to the coordinate sequence of the seal content area to obtain two coordinate sequences representing the inner and outer arcs respectively. Then, the circle fitting calculation method is applied to obtain the center coordinates and radius of the inner and outer arcs, as well as the equation of the circle fitted by the inner and outer arcs.
[0073] The above post-processing methods specifically include:
[0074] Divide the border of the arc-shaped seal content area into two curves, upper and lower. Find the point at the exact midpoint of these two curves. Calculate the angle between the line connecting the center of the circle and the two points in the horizontal and vertical directions, respectively. Obtain the average value of the angles in the horizontal and vertical directions. Based on these two angles, rotate the entire seal to the positive direction and obtain the rotated coordinate sequence through the coordinate rotation calculation method. Classify each point into the inner arc or outer arc coordinate list according to the distance relationship between each point and the current center of the circle. Finally, obtain the coordinate sequences of the inner and outer arcs respectively.
[0075] "Upright" refers to the main content area of the seal opening downwards, being almost symmetrical from left to right, with the text facing upwards and not upside down; this direction is considered the "upright" direction.
[0076] Using the coordinate sequence of the inner and outer arcs, the fitted center coordinates and radius, and the original seal image as inputs to the seal content correction method module, the size of the output rectangular image frame of the seal content correction method module is set. Each pixel position in the output image is mapped to the position in the seal content area through the fitted circle equation. Based on the correspondence between the pixel positions in the output rectangular image and the seal content area, the value of each pixel in the output rectangular image frame is calculated using an interpolation method.
[0077] The specific process for correcting circular arc curves includes:
[0078] The width and height of the output image are preset. Pixels are selected sequentially in row and column order in the output image. The proportions of the horizontal and vertical positions of the pixels are calculated. Using the above proportions, based on the equation of the fitted circle and the coordinates of the start and end points of the inner and outer arcs, a point matching the pixel in the output image is determined in the seal content area. Using an interpolation algorithm and the pixel value information based on the seal content area, the pixel value corresponding to that pixel in the output image is calculated.
[0079] The pixel value of each pixel in the output image is calculated, and the corrected horizontal rectangular text line image of the seal content is finally obtained, resulting in a horizontal seal content image.
[0080] The steps for recognizing seal content include:
[0081] A character recognition algorithm is applied to a horizontal image of the seal content to obtain the text content within the image.
[0082] Specifically, the seal content recognition algorithm is executed sequentially on all the corrected horizontal seal images, and the output is the text content in the horizontal seal content image.
[0083] The following combination Figures 1 to 7 The seal content recognition method of the present invention will be further described in detail.
[0084] Please see Figure 1 , Figure 1 This is a flowchart of the seal content recognition method in this invention. For example... Figure 1 As shown, the seal content recognition method of this invention includes the following steps:
[0085] S100: Obtain the input file and convert the original input file into standard input image data.
[0086] The original file can be in various formats such as PNG, JPEG, TIFF, and PDF. By standardizing the format of the original file, if it is an image file, it can be read directly. If it is a document file such as PDF and contains multiple pages, the pages need to be split and converted into images before reading, thus achieving the universality of the method of this invention for various types of input files.
[0087] S200. Perform a stamp detection and classification algorithm on the input image to detect stamps present in the image and obtain the position coordinates and category of the stamps.
[0088] Specifically, a seal detection and classification algorithm is used to detect and classify seals in the input image. Supported seal categories include square, rhombus, circle, and ellipse. Location information refers to the coordinates of four points within the rectangular region containing the seal. The seal detection algorithm utilizes mainstream object detection algorithms, which can be broadly categorized into two types: One-Stage and Two-Stage algorithms. The former is represented by the YOLO (You Only Look Once) series, which directly predicts the target's location and category using image features extracted by the network. While fast, its accuracy is difficult to guarantee. The latter is represented by the R-CNN series, which first uses image features to generate candidate boxes for object detection, then classifies and regresses these boxes to obtain the final detection result. While slightly slower, it offers higher accuracy. To ensure the accuracy of seal location detection and classification, this invention uses a Two-Stage object detection algorithm.
[0089] S300. Use the seal position coordinates to extract the seal image region from the input image, and perform a seal content detection algorithm on the seal image region to obtain the seal content region.
[0090] Specifically, the seal area is cropped from the original image using the seal's position coordinates. A seal content detection algorithm is then applied to this area to obtain the seal content region. Different seal content detection algorithms are used for different types of seals. These algorithms are divided into horizontal text detection algorithms and arbitrary shape text detection algorithms. Finally, one or more coordinate sequences representing the seal content region are obtained, with each coordinate sequence representing a content region of the seal.
[0091] S400. Different correction methods are used for different shaped seal content areas to obtain a horizontal seal content image.
[0092] S500 performs text recognition on the horizontal image of the seal content to obtain the text content of the seal.
[0093] Specifically, the text recognition method uses the CRNN horizontal text line image recognition algorithm. The generated stamp text image annotation data is used to fine-tune a general text recognition model that has been trained on a large number of scene texts to obtain a text recognition model that is more suitable for the real application scenario of this invention. The text image is input into the above text recognition model to obtain the recognition result.
[0094] As one embodiment of the present invention, such as Figure 2 As shown, step S200 specifically includes:
[0095] S210. Obtain the input image, use the backbone network of the detection model to extract image features, and construct a candidate box set.
[0096] Specifically, the processed image data is input into the seal detection module, and the detection model backbone network is used to extract image features and obtain feature maps. Based on the graphic proportions and sizes of square, rhomboid, elliptical and circular seals, a multi-scale and multi-proportion candidate box is set for each point in the feature map to obtain multiple candidate boxes.
[0097] S220. Perform binary classification and regression on the candidate boxes, filter out some candidate boxes, perform candidate box-feature map alignment on the remaining candidate boxes, extract the features of each candidate box, and finally perform multi-class classification and bounding box coordinate regression on the above candidate boxes to obtain the position information and category information of the detected seal.
[0098] Specifically, the RPN network is used to perform binary classification and regression on the above candidate boxes, filtering out some candidate boxes and updating other candidate boxes according to the regression results. Candidate box-feature map alignment is performed on the remaining candidate boxes to extract the features of each candidate box. Finally, multi-classification and bounding box coordinate regression are performed on the above candidate boxes to obtain the location and category information of the detected seal.
[0099] S230. Perform multi-classification and boundary coordinate regression on the above filtered candidate boxes to obtain the target location coordinates and category that are determined to be seals.
[0100] In the seal content detection step, the input seal image can be square or rhomboid, and its text content is considered to be horizontally distributed; the horizontal seal content area to be detected is the horizontal seal content area. The input seal image can also be circular or elliptical, and its text content is considered to be mainly curved, arc-shaped text areas; the curved seal content area to be detected is the curved seal content area. As an embodiment of the present invention, the seal content detection includes the following steps:
[0101] S310. Based on the position coordinates of the seal, crop the image area that mainly contains the seal from the original image.
[0102] S310. Determine the type of seal corresponding to the position coordinates. This can be obtained through the seal detection and classification algorithm.
[0103] like Figure 3 As shown, if the stamp type is square or rhombus, then perform the following steps S330-S331:
[0104] S330. Execute the horizontal text detection algorithm and input the seal image region into the horizontal text detection model.
[0105] S331. Obtain the coordinate sequence of the rectangular text box representing the content area of the seal in the seal image area.
[0106] Specifically, the horizontal text detection model is trained based on sample data of text detection in normal documents, and the position coordinates of the rectangular text box are the coordinates of its four vertices.
[0107] like Figure 4 As shown, if the stamp type is round or oval, then perform the following steps S340-S341:
[0108] S340. Execute the arbitrary shape text detection algorithm and input the seal image region into the arbitrary shape text detection model.
[0109] S341. Obtain the polygon coordinate sequence representing the content area of the seal in the seal image area.
[0110] Specifically, the arbitrary shape text detection model is obtained by pre-training on a large number of content detection annotation samples of generated seals and fine-tuning the training on a small number of content detection annotation data of various real seals. The polygon coordinate sequence is the coordinate sequence of n points on the boundary of the seal content area, where n is a preset constant.
[0111] In the stamp content area correction step, the input stamp content area may be rectangular, requiring rotation and correction to a horizontal orientation to obtain a horizontal stamp content image; the input text box image may also be arc-shaped, requiring straightening of the arc-shaped stamp content area into a horizontal stamp content image. As an embodiment of the present invention, stamp content area correction includes the following steps:
[0112] S410. Obtain the input sequence of coordinates of the seal content area and the original image data.
[0113] S420. Determine the shape of the input stamp content area. By comparing the ratio of the area of the polygon formed by the coordinate sequence to the area of its smallest generated rectangle with a set threshold, the input stamp content area can be classified as rectangular or arc-shaped.
[0114] like Figure 5 As shown, if the shape of the stamp content area is rectangular, then perform the following steps S430-S431:
[0115] S430. Execute the rectangular text box correction method to calculate the angle required to rotate the input rectangular text box to the horizontal direction.
[0116] S431. Using rotation angle and affine transformation, rotate the tilted rectangular text box to the horizontal position.
[0117] like Figure 6 As shown, if the text box type is arc-shaped, then perform the following steps S440-S444:
[0118] S440. Based on the characteristics of the content area of circular and elliptical seals, execute a seal content area correction algorithm based on arc curves.
[0119] S441. Execute the post-processing algorithm for the arc-shaped seal content area, rearrange all coordinate points, and expand all coordinate points outward to make the area included in the coordinate point sequence better cover the required seal content. The final coordinate point sequence starts from the top left corner of the first character in the forward reading order of the seal content area and rotates clockwise to the bottom left corner of the first character in the forward reading order.
[0120] S442. Using the processed coordinate sequence and calculation method described above, the coordinate sequence is divided into two sequences, corresponding to the inner and outer arcs of the circular arc curve respectively. The equations of the circles corresponding to the inner and outer arc curves are calculated using the two coordinate sequences based on the least squares method. The equations include the center coordinates and radius of the circle.
[0121] S443. The straightening and correction are completed using the equation of the fitted circle, the coordinate sequence, and the original image of the seal content area. The specific process of correction is as follows: the width and height of the output image are preset, and pixels are selected in the output image in row and column order. The proportions of their horizontal and vertical positions are calculated. Using the above proportions, based on the equations of the inner and outer arc curves, the points in the seal content area that match the pixels in the output image are found. The pixel values of the pixels in the output image are calculated using the interpolation method and the pixel value information of the seal content area. Finally, the corrected horizontal image of the seal content is obtained.
[0122] An example of seal content detection and correction, with a real-world application process as follows: Figure 7 As shown, it specifically includes:
[0123] The content area of the seal was detected using an arbitrary shape text detection method.
[0124] A method based on arc curves is used to correct the content area of a circular stamp. For the content area determined to be circular, a calculation method is used to obtain the coordinate sequences belonging to the inner and outer arcs respectively. These coordinate sequences are then used to fit the inner and outer arc curves, obtaining the equations of the circles corresponding to the inner and outer arc curves. The fitted inner and outer arc curves are visualized as follows: Figure 5 As shown by the blue and green lines, the red coordinates are the coordinates output by the arbitrary shape text detection algorithm;
[0125] In the output image of a preset size, pixels are selected sequentially in row and column order. The correspondence between the pixels and the pixels in the seal content area is obtained through the equations of inner and outer arc curves. The pixel values of the pixels in the output image are calculated using interpolation methods, and finally, the corrected horizontal seal content image is obtained.
[0126] The rectangular text box correction method is used on the rectangular stamp content area. The required rotation angle is obtained by calculation. The angle and affine transformation method are used to rotate the tilted rectangular text area back to the horizontal direction, and finally the corrected result is obtained.
[0127] Specifically, in Figure 7 In the real-world example, the arc-shaped seal content area "1234 Futures Co., Ltd." is corrected using an arc curve-based correction algorithm, while the rectangular seal content areas "Futures Brokerage Contract Account Opening Special Seal" and "1" are corrected using a rectangular text box correction algorithm.
[0128] This invention provides a method for recognizing seal content in images. It uses object detection, text detection, text recognition, and other related methods to achieve seal detection, classification, content detection, and content recognition in images. Based on the text morphological characteristics of circular and elliptical seals, an arc curve correction method adapted to the correction of curved text in these circular shapes is designed. Moreover, it achieves good recognition accuracy with relatively little data resources, greatly reducing the investment of manual resources in seal recognition and verification in real-world scenarios, improving the office efficiency of relevant departments, and effectively solving practical problems.
[0129] Compared with existing technologies, the advantages of this invention are: the streamlined process of seal detection, content extraction, correction, and recognition reduces the need for real seal annotation data while still achieving high accuracy in detection, classification, content extraction, and recognition. Furthermore, considering the complex backgrounds and noise interference often present in real-world scenarios, the generated data used in model training is designed to closely resemble reality, ensuring the adaptability and effectiveness of this invention across various scenarios. This invention effectively solves the problem of seal content recognition in scenarios such as contract entry systems, inventory management, and invoice reimbursement, comparing and verifying the names of contract or invoice entities with the seal content, saving significant labor costs and improving enterprise office efficiency.
[0130] To achieve the above objectives, the present invention also proposes a seal content recognition system, the system including a memory, a processor, and a seal content recognition program stored on the processor. When the seal content recognition program is run by the processor, it executes the steps described in the above embodiments, which will not be repeated here.
[0131] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a seal content recognition program. When the seal content recognition program is run by a processor, it executes the steps described in the above embodiments, which will not be repeated here.
[0132] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.
Claims
1. A method for recognizing the content of a seal, characterized in that, The method includes the following steps: Seal position detection steps: acquire an image with a seal, execute a seal detection and classification algorithm to detect and locate the seal in the image, and obtain the position coordinates and category of the seal in the image. The seal categories include circle, ellipse, rhombus and square. Seal content detection steps: The image area of the seal is cropped according to the position coordinates of the seal, and the seal content detection algorithm is executed on the image area of the seal to obtain the seal content area represented by a list of polygon coordinates; Seal content area correction steps: Sort the polygon coordinates, and for seal content areas of the seal category of circle and ellipse, execute the seal content area correction algorithm based on arc curve to correct the circle or ellipse seal content area into a rectangle to obtain the corrected seal content area. Seal content recognition steps: For the content area of a rhombus or square seal, or the corrected content area of the seal, execute the seal content recognition algorithm and output the text content of the seal; The seal content detection steps include: Using all the seal images and corresponding category information obtained in the seal detection step as input information for the seal content detection step, content detection is performed on each seal according to the corresponding detection method based on the category. In the case where the seal is square or rhomboid, a horizontal text detection method is used to perform a horizontal text detection algorithm on the seal image to obtain the seal content area. When the seal category is circular or elliptical, an arbitrary shape text detection method is used to perform an arbitrary shape text detection algorithm on the seal image to obtain the seal content area; When the seal category is circular or elliptical, the step of performing an arbitrary shape text detection algorithm on the seal image to obtain the seal content area includes: For seal images classified as circular or elliptical, a text detection algorithm based on instance segmentation is executed to obtain a pixel classification probability map and a seal image feature map. A large probability threshold is selected to ensure that the initial seal content regions are all inside the real seal content regions. A binarization method is performed on the pixel classification probability map to obtain a binarized image. A seal content region bounding box generation algorithm is then performed on the binarized image to obtain multiple initial bounding box coordinate sequences for the seal content regions. Using multiple initial bounding box coordinate sequences of the seal content region, feature vectors corresponding to each coordinate are extracted from the seal image feature map to obtain feature representations corresponding to multiple text instances. The feature representations corresponding to multiple text instances are input into a seal content region expansion algorithm based on a deep neural network model to obtain the predicted displacement distance and displacement direction of each coordinate point. The displacement distance and displacement direction of each coordinate point are applied to the corresponding coordinate point to obtain a more accurate bounding box coordinate sequence of the seal content region compared to directly binarizing based on a preset threshold. The steps for correcting the content area of the seal include: If the stamp content area obtained by the horizontal document detection method is determined to be rectangular, a rectangular rotation and leveling algorithm is executed to correct the stamp content area to the horizontal direction, thereby obtaining a horizontal stamp content image. For the coordinate sequence of the stamp content area returned by the arbitrary shape text detection method, if it is determined that the coordinate sequence of the stamp content area forms a polygon to be classified, the polygon shape judgment method is executed on the stamp content area. The area of the area is obtained through the coordinate sequence of the stamp content area. At the same time, the minimum generated rectangle including this stamp content area is generated, and the area of the minimum generated rectangle is calculated. The ratio of the area of the stamp content area to the area of the corresponding minimum generated rectangle is calculated and compared with a set threshold. If it is greater than the threshold, the shape of the stamp content area is determined to be a rectangle; if it is less than the threshold, the shape of the stamp content area is determined to be an arc. For the rectangular stamp content area, perform a rectangular rotation and leveling algorithm to adjust it to the horizontal direction, resulting in a horizontal stamp content image; For the arc-shaped seal content area, a seal content area correction algorithm based on the arc curve is executed. The special characteristic of the curved text area of the seal is that it is straightened and corrected to obtain a horizontal seal content image.
2. The seal content recognition method according to claim 1, characterized in that, The seal position detection step includes: Obtain the feature map of the input image and construct multiple candidate boxes that may contain the seal; The candidate boxes are subjected to binary classification and regression, and the filtered portion is determined to be the background. Perform candidate box-feature map alignment on the remaining candidate boxes to obtain the feature representation of the corresponding region on the image feature map. Then, perform multi-classification and border position offset regression on the remaining candidate boxes to obtain the stamp position coordinates and category.
3. The seal content recognition method according to claim 2, characterized in that, The step of correcting the content area of the seal also includes: The post-processing method is executed to rotate the seal image to the positive direction according to the coordinate sequence of the seal content area. At the same time, the coordinate sequence is rearranged and appropriately expanded to obtain the coordinate sequence of the seal content area, which rotates clockwise from the upper left corner of the semantic seal content. The inner and outer arc coordinate sequence calculation algorithm is executed on the coordinate sequence of the seal content area to obtain two coordinate sequences representing the inner and outer arcs respectively. Then, the circle fitting calculation method is executed to obtain the center coordinates and radius of the inner and outer arcs, as well as the equation of the circle fitted by the inner and outer arcs. The width and height of the output image are preset. Pixels are selected in the output image in row and column order. The proportion of the horizontal and vertical positions of the pixels is calculated. Using the proportion, based on the fitted equation of the circle and the coordinates of the start and end points of the inner and outer arcs, a point matching the pixel in the output image is determined in the seal content area. Using an interpolation algorithm, the pixel value corresponding to the pixel in the output image is calculated based on the pixel value information of the seal content area. Calculate the pixel value of each pixel in the output image to obtain the horizontal stamp content image.
4. The seal content recognition method according to claim 3, characterized in that, The seal content recognition step includes: A character recognition algorithm is performed on the horizontal seal content image to obtain the text content in the seal content image.
5. A seal content recognition system, characterized in that, The system includes a memory, a processor, and a stamp content recognition program stored on the processor, the stamp content recognition program being executed by the processor to perform the steps of the method as described in any one of claims 1 to 4.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a seal content recognition program, which, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Intelligent seal detection and identification method
CN110728277A
