Seal text recognition method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a seal character recognition method, an electronic device and a storage medium. First, the shape category information, the position information and the position information of the seal key points of a seal are obtained by performing seal detection on an object picture including the seal. Then, the original seal image is obtained from the object picture according to the position information of the seal, and the original seal image is subjected to noise reduction processing to obtain a seal image. Finally, the seal image is subjected to image processing using the position information of the seal key points of the seal of different shape categories to obtain one or more to-be-recognized character images, and the one or more to-be-recognized character images are subjected to character recognition to obtain a character recognition result of the seal. Thus, a new seal character recognition mode is provided, which can effectively improve the recognition accuracy of seal characters and support character recognition of seals of various styles, various colors and various shapes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for recognizing characters on a seal, an electronic device, and a storage medium. Background Technology

[0002] For documents bearing seals, the text on the seal is a crucial piece of information that needs to be accurately identified. In practical applications, seals come in a wide variety of shapes, including but not limited to: round, square, and oval seals; the layout and curvature of the text, the amount of ink, the angle at which the seal is stamped, the seal color, and the text in the area below the seal are also complex and varied; these factors increase the difficulty of seal text recognition and affect the accuracy of seal text recognition. Summary of the Invention

[0003] This application provides a method for recognizing characters on a seal, an electronic device, and a storage medium to improve the accuracy of character recognition on a seal.

[0004] This application provides a method for recognizing characters on a seal, comprising: performing seal detection on an object image including a seal to obtain a seal detection result, the seal detection result including seal shape category information, position information, and position information of key points of the seal; obtaining an original seal image from the object image based on the seal position information; performing noise reduction processing on the original seal image to obtain a seal image; performing image processing on the seal image based on the shape category information and the position information of key points of the seal to obtain at least one image to be recognized; and performing character recognition on the at least one image to be recognized to obtain a character recognition result for the seal.

[0005] This application also provides a method for recognizing characters on a seal, applied to a cloud server. The method includes: receiving a seal recognition request sent by a terminal device, the seal recognition request carrying an object image including a seal; responding to the seal recognition request, performing seal detection on the object image to obtain a seal detection result, the seal detection result including seal shape category information, location information, and location information of key points on the seal; obtaining an original seal image from the object image based on the seal location information; performing noise reduction processing on the original seal image to obtain a seal image; performing image processing on the seal image based on the shape category information and the location information of key points on the seal to obtain at least one image to be recognized; performing character recognition on the at least one image to be recognized to obtain a seal character recognition result; and returning the seal character recognition result to the terminal device.

[0006] This application also provides an electronic device, including: a memory and a processor; the memory for storing a computer program; and the processor coupled to the memory for executing the computer program to perform steps in the seal character recognition method.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the seal character recognition method. Attached Figure Description

[0008] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0009] Figure 1 An application scenario diagram provided for an embodiment of this application;

[0010] Figure 2 A flowchart of a seal character recognition method provided in this application embodiment;

[0011] Figure 3 Three exemplary seals provided for embodiments of this application;

[0012] Figure 4 An exemplary diagram illustrating the rotation process of a square seal, provided for embodiments of this application;

[0013] Figure 5 An exemplary circular seal image processing diagram provided for embodiments of this application;

[0014] Figure 6 An exemplary image processing diagram of an oval seal provided for embodiments of this application;

[0015] Figure 7a Exemplary noisy stamp images and clean stamp images provided for embodiments of this application;

[0016] Figure 7b A schematic diagram of a model training principle provided in an embodiment of this application;

[0017] Figure 7c A flowchart of another seal character recognition method provided in the embodiments of this application;

[0018] Figure 8 A flowchart of another seal character recognition method provided in the embodiments of this application;

[0019] Figure 9 This is a schematic diagram of the structure of a seal character recognition device provided in an embodiment of this application;

[0020] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship. Furthermore, in the embodiments of this application, "first," "second," "third," etc., are only used to distinguish the content of different objects and have no other special meaning.

[0023] In practical applications, seals come in a variety of shapes, including but not limited to: round seals, square seals, and oval seals; the layout and curvature of the seal text, the amount of ink, the angle at which the seal is stamped, the seal color, and the text in the area below the seal are also complex and varied; the above factors increase the difficulty of seal text recognition and affect the accuracy of seal text recognition.

[0024] Based on the above, embodiments of this application provide a method, electronic device, and storage medium for recognizing characters on a seal. In this embodiment, firstly, seal detection is performed on an object image including a seal to obtain the seal's shape category information, location information, and location information of key points. Next, an original seal image is obtained from the object image based on the seal's location information, and noise reduction processing is applied to the original seal image to obtain a seal image. Finally, image processing is performed on the seal images using the location information of key points for seals of different shape categories to obtain one or more images to be recognized for character recognition. Character recognition is then performed on the one or more images to be recognized for character recognition to obtain the seal's character recognition result. Therefore, a new method for recognizing characters on a seal is provided, which can effectively improve the accuracy of seal character recognition and supports character recognition for seals of various styles, colors, and shapes.

[0025] Figure 1This diagram illustrates an application scenario provided by an embodiment of this application. In practical applications, the need to recognize the text on stamps is very common. For example, in the government sector, there is a large amount of document data that needs to be processed, and the stamps on these documents are one of the important pieces of information. Recognizing the text on stamps on documents is of great value to the government sector. Another example is in invoice reimbursement scenarios, where it is necessary to recognize the text on stamped invoices. Yet another example is in contract management scenarios, where it is necessary to recognize the text on stamped contracts.

[0026] When users require text for a seal, see [link / reference]. Figure 1 As shown in ①, a scanner is used to scan a paper document bearing a seal, obtaining an image of the object including the seal. Alternatively, an image acquisition device can be used to photograph the paper document bearing the seal, obtaining an image of the object including the seal. The image acquisition device can be any device with image acquisition capabilities. For example, image acquisition devices can be classified according to the structural characteristics of their sensors as area scan cameras and line scan cameras. Furthermore, image acquisition devices can be classified according to different screen resolutions as standard definition cameras and high definition cameras. Still others can be classified according to signal type as analog cameras and digital cameras.

[0027] See Figure 1 As shown in ②, the scanner sends an image of the object, including the seal, to the user's terminal device. See also Figure 1 As shown in ③, the user triggers the terminal device to send a seal text recognition request to the cloud server. The seal text recognition request carries an image of the object including the seal. See also Figure 1 As shown in ④, in response to a seal text recognition request, the cloud server performs seal detection on the image containing the seal. The seal detection result includes the seal's shape category information, location information, and the location information of the seal's key points. See also Figure 1 As shown in ⑤, the cloud server retrieves the original seal image from the object image based on the seal's location information. (See also...) Figure 1 As shown in ⑥, the cloud server performs noise reduction processing on the original seal image to obtain the seal image; see also Figure 1 As shown in ⑦, the cloud server performs image processing on the seal image based on shape category information and the location information of key points on the seal, obtaining at least one image to be recognized for text; see also Figure 1 As shown in ⑧, the cloud server performs text recognition on at least one image to be recognized, obtaining the text recognition result of the seal. See also Figure 1 As shown in diagram ⑨, the cloud server sends the text recognition results of the seal to the user's terminal device to meet the user's seal text requirements. This automatically completes the seal text recognition task, improving the accuracy and efficiency of seal text recognition.

[0028] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0029] Figure 2 This is a flowchart illustrating a seal character recognition method provided in an embodiment of this application. The method can be executed by a seal character recognition device, which can consist of software and / or hardware, and is generally configured in an electronic device.

[0030] See Figure 2 The method may include the following steps:

[0031] 201. Perform seal detection on the object image containing the seal to obtain the seal detection results. The seal detection results include the seal's shape category information, location information, and location information of the seal's key points.

[0032] 202. Obtain the original seal image from the object image based on the seal's position information.

[0033] 203. Perform noise reduction processing on the original seal image to obtain the seal image.

[0034] 204. Based on the shape category information and the location information of the key points of the seal, perform image processing on the seal image to obtain at least one image to be recognized by text.

[0035] 205. Perform character recognition on at least one image to be recognized to obtain the character recognition result of the seal.

[0036] In this embodiment, the object for which seal text recognition is required can be any paper or electronic document bearing a seal, such as, but not limited to, contract texts, invoices, policy promotional documents, and product brochures.

[0037] In this embodiment, an image acquisition device can be used to acquire an image of the object for which seal text recognition is required, or a scanner can be used to scan the object for which seal text recognition is required to obtain an image of the object including the seal. Additionally, when the object for which seal text recognition is performed is a PDF (Portable Document Format) file, the PDF file can be converted from PDF format to image format to obtain an image of the object including the seal.

[0038] In this embodiment, a seal is used as the target detection object. Seal detection is performed on an image containing the seal to obtain the seal detection result. The seal detection result includes, but is not limited to, the seal's shape category information, location information, and the location information of the seal's key points. It is understood that the seal detection includes a key point detection function, which can detect the location information of the seal's key points.

[0039] In this embodiment, shape category information is used to indicate the shape category to which the seal belongs. Shape category information includes, but is not limited to, square, circle, and ellipse.

[0040] A square seal is rectangular in shape, and text is usually embedded around its perimeter. The text may be arranged vertically from top to bottom or horizontally from left to right.

[0041] A circular seal is circular in shape. The upper arc of the seal typically contains the seal's name, arranged in a circular pattern from left to right. The central area may contain various shapes such as a five-pointed star or triangle. The lower area may contain the seal's name or a numerical designation. The name is arranged horizontally from left to right, while the designation is arranged in a circular pattern from left to right. The seal's name refers to the name of the entity to which the seal belongs, such as a company, institution, organization, or individual. The upper arc of the seal is the arc between the circumference of the first character in the seal's name and the circumference of the last character. The lower area refers to the area of ​​the seal excluding the upper arc and the central area. Of course, the content embedded in the upper arc, central area, and lower area may vary depending on the specific usage.

[0042] The upper arc of an oval seal typically contains the seal's name in a circular pattern from left to right. The central area may contain the seal's name in a horizontal pattern from left to right, or a numerical designation. The lower area may contain a circular numerical designation from left to right. The upper arc of an oval seal refers to the arc between the circumference of the first character in the seal's name and the circumference of the last character. The lower area refers to the area of ​​the oval seal excluding the upper arc and the central area. Of course, the content embedded in the upper arc, central area, and lower area may vary depending on the specific usage scenario.

[0043] In this embodiment, the position information of the seal refers to the position information of the seal in the object image. Based on the position information of the seal, the image area where the seal is located can be located in the object image.

[0044] In this embodiment, the key points of the seal refer to the key points related to the seal, and the location information of the key points of the seal refers to the location information of the key points of the seal in the object image. The key points of the seal can be located from the object image based on the location information of the key points of the seal.

[0045] In this embodiment, see Figure 3For square seals, the focus is on key points such as the lower right and lower left corners, and the center point of the seal. The lower right and lower left corners of a square seal can refer to the left and right vertices on the lower boundary of the detection frame surrounding the seal. For round and oval seals, the focus is on the center point, the circumference point defining the first character of the seal's name, and the circumference point defining the last character of the seal's name.

[0046] In this embodiment, there are no restrictions on the method of seal detection. For example, seal detection can be performed manually. Alternatively, seal detection can be performed automatically by sequentially performing grayscale conversion, Gaussian smoothing, edge detection, and Hough transform on the image containing the seal.

[0047] Alternatively, to improve the accuracy of seal detection results, any object detection algorithm can be used to perform seal detection on the image containing the seal, obtaining the seal detection results. For example, seal detection can be performed based on the SSD (Single Shot MultiBox Detector) detection framework or the RefineDet object detection framework. SSD is a single-stage object detection algorithm that extracts features through a convolutional neural network, taking different feature layers for detection output; SSD is a multi-scale detection method. RefineDet is based on a feedforward convolutional network that generates a fixed number of bounding boxes and scores representing different categories of objects present in these boxes, producing the final result through non-maximum suppression (NMS).

[0048] Alternatively, to improve the accuracy of seal detection results, a pre-trained seal detection model can be used to perform seal detection on object images containing seals.

[0049] Specifically, the seal detection model is trained using a massive amount of sample object images and their annotation results. The annotation results include, but are not limited to, information about the shape category, location, and key points of the seal in the sample object images. The trained seal detection model can accurately and comprehensively detect seal information. The seal detection model can include, but is not limited to, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM).

[0050] During the model training phase, the model is trained iteratively multiple times. After each training iteration, the loss value is calculated based on the stamp detection results and annotation results output by the stamp detection model. The model parameters are adjusted to minimize the loss value. The adjusted stamp detection model is then used as the next model to be trained, and the next training iteration is performed using sample object images and their annotation results, until the training termination condition is met. The training termination condition can be either reaching a specified number of training iterations or the loss value reaching its minimum.

[0051] During the model training phase, various numerical calculations, such as summation, averaging, or accumulation, are performed on the loss values ​​calculated based on the shape category information output by the stamp detection model and the shape category information in the annotation results, the loss values ​​calculated based on the position information of the stamp output by the stamp detection model and the position information of the stamp in the annotation results, and the loss values ​​calculated based on the position information of the stamp key points output by the stamp detection model and the position information of the stamp key points in the annotation results. The loss functions used to calculate the loss values ​​include, but are not limited to: the logarithmic loss function, the L1 distance loss function, the cross-entropy loss function, and the Focal loss function used to address data imbalance problems.

[0052] In some optional embodiments, to improve the accuracy of seal detection results, the seal detection model can be a YOLOv5-face model, an open-source object detection model. While the YOLOv5-face model was first applied to the field of face detection, its idea of ​​simultaneously detecting target location and key points is applicable in many fields. Therefore, the YOLOv5-face model has been extended and optimized to simultaneously detect the seal's location, key points, and category.

[0053] In this embodiment, after performing seal detection on the object image including the seal, the image region where the seal is located is located from the object image based on the detected seal's position information, and this image region is used as the original seal image. Compared to the object image, the original seal image removes the image region other than the seal. Using the original seal image can effectively reduce the difficulty of seal text recognition and improve the accuracy of seal text recognition.

[0054] In this embodiment, the original seal image may contain some noise. In order to reduce the impact of noise on the recognition accuracy of seal characters, the original seal image can be denoised to obtain the seal image.

[0055] In this embodiment, after obtaining the denoised seal image, the seal image is processed according to the shape category information and the position information of the key points of the seal to obtain at least one image to be recognized by text. Text recognition is performed on the at least one image to be recognized by text to obtain the text recognition result of the seal, thereby realizing the support for text recognition of various seals.

[0056] In this embodiment, the method of text recognition is not limited; for example, various existing text recognition technologies can be used. Further optionally, to improve the accuracy of text recognition, a pre-trained text recognition model can be used. This text recognition model is obtained by training a machine learning model using multiple sample images labeled with text information.

[0057] In this embodiment, text recognition of square seals can be supported. Considering the significant randomness of the seal's stamping angle, to improve the accuracy of text recognition for square seals, the seal image is aligned through image rotation. Therefore, when processing the seal image based on shape category information and the position information of key points on the seal to obtain at least one image for text recognition, if the shape category information is square, the seal image is rotated based on the position information of the key points on the seal to obtain at least one image for text recognition.

[0058] Further, optionally, to improve the accuracy of character recognition for square seals, if the shape category information is square, one implementation method for rotating the seal image based on the position information of the seal's key points to obtain at least one image for character recognition is as follows: If the shape category information is square, and the seal's key points include the lower right corner vertex, the lower left corner vertex, and the center point of the seal, then based on the position information of the lower right and lower left corner vertices, determine the first angle between the direction of the line connecting the lower right and lower left corner vertices and the horizontal direction; using the center point of the seal as the rotation center, rotate the seal image by the first angle to obtain the image for character recognition. For ease of understanding and distinction, the angle between the direction of the line connecting the lower right and lower left corner vertices and the horizontal direction is referred to as the first angle.

[0059] Specifically, see Figure 4 The angle between the line connecting the lower right and lower left vertices and the horizontal direction is θ. The seal image before rotation is rotated by θ with the center point of the seal as the rotation center. The angle between the line connecting the lower right and lower left vertices of the rotated seal image and the horizontal direction is 0 degrees, that is, the rotated seal image is straightened. The straightened seal image is the image to be recognized for text recognition.

[0060] In this embodiment, text recognition of circular seals can be supported. To improve the accuracy of text recognition of circular seals, image processing such as image rotation, partitioning, or polar coordinate transformation is performed on the seal image. Based on this, when image processing is performed on the seal image according to shape category information and the position information of key points on the seal to obtain at least one image to be recognized, if the shape category information is circular, then image rotation, polar coordinate transformation, and masking processing are performed on the seal image according to the position information of key points on the seal to obtain at least one image to be recognized.

[0061] Further optionally, to improve the accuracy of character recognition for circular seals, if the shape category information is circular, then based on the position information of the seal's key points, image rotation, polar coordinate transformation, and masking are performed on the seal image to obtain at least one image for character recognition. One implementation is as follows: if the shape category information is circular, then based on the position information of the seal's center point, the first circumferential point used to define the first character in the seal's main name, and the second circumferential point used to define the last character in the seal's main name, the first sector area excluding the seal's main name in the seal image and the area in the copied seal image are determined. The second sector-shaped area includes the name of the main body of the seal. The process involves copying the seal image, masking the first sector-shaped area of ​​the seal image to obtain a masked seal image, and then sequentially rotating and transforming the masked seal image to obtain a first image to be recognized by text. The second sector-shaped area of ​​the copied seal image is then masked to obtain a masked copied seal image. The masked copied seal image is then rotated to obtain a second image to be recognized by text, and finally, a polar coordinate transformation is performed on the second image to be recognized by text to obtain a third image to be recognized by text.

[0062] In this embodiment, the region is divided based on the positional information of the key points of the seal. As an example, to improve the accuracy of text recognition, when determining the first sector region in the seal image that does not include the name of the seal's main body and the second sector region in the copied seal image that includes the name of the seal's main body, a second included angle between the first and second connecting lines can be determined based on the positional information of the seal's center point, the first circumference point, and the second circumference point. The first connecting line refers to the direction of the line connecting the seal's center point and the first circumference point, and the second connecting line refers to the direction of the line connecting the seal's center point and the second circumference point. The sector region in both the seal image and the copied seal image, with the seal's center point as the vertex and the second included angle as the central angle, is defined as the first sector region. The sector region in the copied seal image, excluding the first sector region, is defined as the second sector region. For ease of understanding and distinction, the included angle between the first and second connecting lines is referred to as the second included angle. It is understandable that the first sector area of ​​both the seal image and the reproduced seal image is a sector area with the center point of the seal as the vertex and the second included angle as the central angle. The second sector area of ​​the seal image is also a sector area excluding the first sector area.

[0063] In this embodiment, when performing masking, the pixel values ​​of the pixels in the area to be masked can be set to a specified value. The specified value can be flexibly set as needed, for example, 0.

[0064] In this embodiment, when performing image rotation processing, the angle between the line connecting the first and second circumference points and the horizontal direction is determined based on the position information of the first and second circumference points. The stamp image is rotated by the angle with the center point of the stamp as the rotation center until the line connecting the first and second circumference points is parallel to the horizontal direction.

[0065] In this embodiment, polar coordinate transformation refers to transforming an image between a Cartesian coordinate system and a polar coordinate system, thereby transforming a circular region into a rectangular region. After polar coordinate transformation, the annular text on the circular stamp can be arranged horizontally on the new image, facilitating operations such as text recognition.

[0066] It is worth noting that performing image rotation before polar coordinate transformation ensures that a character will not be split in half during the subsequent text unfolding process using polar coordinate transformation, thus guaranteeing the accuracy of text recognition.

[0067] Specifically, considering that the characters in different areas of a circular seal have different shapes and curvatures, it is necessary to identify each area separately. Figure 5 Images (a), (b), and (c) illustrate the process of partitioning and masking a circular stamp. First, based on the lines connecting the stamp's center point and the first circumference point, and the lines connecting the stamp's center point and the second circumference point, the process can be... Figure 5 The seal image shown in (a) is divided into a first sector and a second sector. Figure 5 The second sector area in the seal image shown in (a) is masked to obtain... Figure 5 The masked stamp image shown in (b) is also shown; additionally, for the... Figure 5 The first sector of the copied seal image shown in (a) is masked to obtain the following result: Figure 5 The masked copy of the stamp image shown in (c) is shown in the image.

[0068] In practical applications, the first sector area may contain circular text that runs along the circumference, for example... Figure 5 In (a) of the text, there is "XXXXXX Unlimited Company"; the second sector area may contain circular text that runs along the circumference, for example... Figure 5 The "123" in (a) of the image; the second sector may also contain horizontal or vertical text, for example... Figure 5 The “simulation chapter” in (a) of the text.

[0069] Therefore, to Figure 5 The masked stamp image shown in (b) is rotated to... Figure 5 The masked seal image shown in (b) is straightened, and then a polar coordinate transformation is performed on the straightened masked seal image to obtain... Figure 5 The first image to be recognized, shown in (e), is used to realize that the circular text "XXXXXXXXX Unlimited Company" is displayed in a horizontal format in the first image to be recognized.

[0070] right Figure 5 The masked copy of the stamp image shown in (c) is rotated to... Figure 5 The masked copy of the stamp image shown in (c) is aligned to obtain... Figure 5 The second image to be recognized is shown in (d) in the figure.

[0071] right Figure 5 The second image to be recognized, shown in (d), is transformed into polar coordinates to obtain... Figure 5 The third image to be recognized is shown in (f) in the figure.

[0072] right Figure 5 The first image to be recognized, shown in (e), can accurately identify "XXXXXXXXX Unlimited Company" using text recognition. Figure 5 The second image to be recognized, shown in (d), can accurately identify "Simulated Chapter". Figure 5The third image to be recognized, shown in (f), can accurately identify "123".

[0073] Furthermore, there are overlapping regions in the second and third images to be recognized. These overlapping regions ensure that no characters are missed. However, they can also lead to the recognition of duplicate characters. For example, when recognizing the second image, all or part of "123" might be recognized; similarly, when recognizing the third image, all or part of "imitation stamp" might be recognized. Duplicate characters can be filtered using flexible, customizable filtering rules. For instance, extensive analysis reveals that the circular text along the circumference in the second sector is primarily numbers, while the horizontal or vertical text is mainly characters other than numbers. Therefore, the filtering rule for the second image is to remove numbers; the filtering rule for the third image is to remove characters other than numbers. Another example is filtering out semantically unclear text. When recognizing the circular text "123" in the second image and the circular text "imitation stamp" in the third image, semantically unclear text is easily identified. In this case, the semantically unclear text is removed.

[0074] In this embodiment, text recognition of elliptical seals can be supported. To improve the accuracy of text recognition for elliptical seals, image processing such as image rotation, size adjustment, partitioning, or polar coordinate transformation is performed on the seal image. Therefore, when image processing is performed on the seal image based on shape category information and the position information of key points on the seal to obtain at least one image to be recognized, if the shape category information is elliptical, then based on the position information of the key points on the seal, image rotation, polar coordinate transformation, and masking are performed on the seal image to obtain at least one image to be recognized.

[0075] Further optionally, to improve the accuracy of text recognition for elliptical seals, if the shape category information is elliptical, then based on the position information of the seal's key points, the seal image is subjected to image rotation processing, polar coordinate transformation, and masking processing to obtain at least one image to be recognized. One implementation is as follows: If the shape category information is elliptical, based on the position information of the seal's key points, image rotation and size adjustment operations are sequentially performed on the seal image to obtain a first circular seal image; a second circular seal image and a third circular seal image are obtained by copying the first circular seal image; based on the position information of the seal's key points in the first circular seal image, the areas to be masked in the first, second, and third circular seal images are determined; the areas to be masked in the first, second, and third circular seal images are respectively masked; the masked second circular seal image is used as the fourth image to be recognized, and the masked first and third circular seal images are respectively subjected to polar coordinate transformation to obtain a fifth and a sixth image to be recognized.

[0076] In this embodiment, the key points of the oval seal include the center point of the seal, the third circumference point used to define the first character in the name of the seal body, and the fourth circumference point used to define the last character in the name of the seal body.

[0077] First, based on the positional information of the third and fourth circumference points, determine the angle between the line connecting the third and fourth circumference points and the horizontal direction. Using the center point of the seal as the rotation center, rotate the seal image by the included angle to align it. In the aligned seal image, the angle between the line connecting the third and fourth circumference points and the horizontal direction is 0 degrees. For example, Figure 6 The seal image shown in (a) has been rotated and aligned to the correct orientation. Figure 6 The image of the seal shown in (b) is shown in the image.

[0078] Next, the oval seal image is resized to a circular seal image. For example, Figure 6 The oval seal image shown in (b) is adjusted to Figure 6 The circular seal image shown in (c) is shown in the image.

[0079] Next, Figure 6 The circular stamp image shown in (c) is copied twice to obtain two circular stamp images. In this way, a total of 3 identical circular stamp images are obtained, which are denoted as the first circular stamp image, the second circular stamp image and the third circular stamp image, respectively.

[0080] Next, the three identical circular seal images are partitioned. Since image rotation and resizing operations alter the positional information of pixels in the image, the circular seal image is partitioned using the positional information of the key points in the first circular seal image. It can be understood that by adjusting the positional information of the key points detected in the seal detection stage according to the rotation matrix corresponding to image rotation and the scaling ratio corresponding to resizing operations, the positional information of the key points in the first circular seal image can be obtained.

[0081] In this embodiment, the upper ring region, lower ring region, and center region can be located based on the positional information of key points in the first circular seal image. See also... Figure 6 As shown in (a), the annular region containing the main name of the seal is divided into an upper annular region including the main name and a lower annular region excluding the main name. The third and fourth circumferential points are the boundary points between the upper and lower annular regions. The annular region containing the main name of the seal is an annular region extending towards the center point of the seal, with the circumference of the circular seal image as its outer circumference. The size of the annular region containing the main name of the seal is related to the size of its inner diameter. The size of the inner diameter of the annular region containing the main name of the seal can be flexibly set as needed, with the goal of ensuring that the upper annular region can cover the main name of the seal. Optionally, to accurately locate the annular region containing the main name of the seal, the seal detection model can be equipped with a function to detect the inner diameter of the annular region containing the main name of the seal, automatically detecting the inner diameter of the annular region containing the main name of the seal. Understandably, during the training phase of the seal detection model, the annotation results of the sample object images are additionally supplemented with the inner diameter of the annular region where the seal's main name is located, and the model is trained with the prediction of the inner diameter of the annular region where the seal's main name is located as one objective.

[0082] In this embodiment, the central region of the seal image refers to the area extending outward from the center point of the seal. The central region of the seal image may not completely overlap with or may partially overlap with the annular region containing the seal's main name. Specifically, the central region of the seal image is the area excluding the annular region containing the seal's main name; in this case, the central region of the seal image does not completely overlap with the annular region containing the seal's main name. Further optionally, to improve the accuracy of text recognition in the central region of the seal image, the central region of the seal image may partially overlap with the annular region containing the seal's main name. See also... Figure 6 As shown in (e), the central region covers part of the lower annular region.

[0083] As an example, when determining the masking area in the first, second, and third circular seal images based on the position information of the key points of the seal in the first circular seal image, the upper ring area including the seal's main name in the first circular seal image can be determined based on the position information of the third and fourth circumference points in the first circular seal image, and the image area in the first circular seal image excluding the upper ring area can be used as the masking area in the first circular seal image; the second circular seal image can be masked based on the position information of the center point of the seal in the first circular seal image. The central region of the seal image, and the image region other than the central region of the second circular seal image, are taken as the masking region in the second circular seal image; wherein, the central region of the second circular seal image does not overlap completely or partially overlaps with the annular region where the seal's main name is located; based on the position information of the third and fourth circumference points in the first circular seal image, the lower annular region of the third circular seal image, excluding the seal's main name, is determined, and the image region other than the lower annular region of the third circular seal image is taken as the masking region in the third circular seal image.

[0084] For example, Figure 6 In the circular stamp image shown in (d), the blackened area is the area to be masked, and the unmasked area is the upper ring area in the circular stamp image. Figure 6 In the circular stamp image shown in (e), the blackened area is the area to be masked, and the unmasked area is the central area of ​​the circular stamp image. Figure 6 In the circular stamp image shown in (f), the blackened area is the area to be masked, and the unmasked area is the lower ring area in the circular stamp image.

[0085] In this embodiment, after masking the areas to be masked in the first, second, and third circular seal images respectively, the masked second circular seal image is used as the fourth image to be recognized by text. Polar coordinate transformation is performed on the masked first and third circular seal images respectively to obtain the fifth and sixth images to be recognized by text.

[0086] For example, since the text in the central area of ​​a circular seal image is mainly horizontal or vertical, it can be directly... Figure 6 The circular stamp image shown in (e) (i.e., the masked second circular stamp image) is used as an image to be recognized as text (i.e., the fourth image to be recognized as text). Since the text in the circular area of ​​the circular stamp image is mainly circular text, it is necessary to... Figure 6 The circular stamp image shown in (d) (i.e., the first circular stamp image after masking) is transformed into polar coordinates to obtain... Figure 6The rectangular seal image shown in (g) serves as the image to be recognized for text recognition (i.e., the fifth image to be recognized for text recognition); Figure 6 The circular stamp image shown in (f) (i.e., the third circular stamp image after masking) is transformed into polar coordinates to obtain... Figure 6 The rectangular stamp image shown in (h) serves as an image to be recognized by text (i.e., the sixth image to be recognized by text).

[0087] In practical applications, multiple images for text recognition may contain overlapping regions, which can ensure that no text is missed. However, overlapping regions can also lead to the recognition of duplicate text. For duplicate text, filtering rules can be flexibly set as needed. For example, based on extensive analysis, if the upper ring region mainly contains Chinese characters, the lower ring region mainly contains numbers, and the central region mainly contains both Chinese characters and numbers, then the filtering rule for the fourth image to be recognized is to retain only Chinese characters and numbers; the filtering rule for the fifth image is to retain only Chinese characters; and the filtering rule for the sixth image is to retain only numbers.

[0088] As can be seen from the above, by utilizing the positional information of the detected key points of the seal, the rotation angle of the seal image can be determined and the seal image can be aligned. Furthermore, the seal image can be partitioned, thereby ensuring good seal text recognition performance and supporting text recognition for various seal styles. In particular, for circular and oval seals, aligning the seal image effectively reduces the probability of a character splitting in two during text unfolding, ensuring high text recognition accuracy.

[0089] In this application embodiment, the style of the seal includes, but is not limited to: an oval or circular seal with text only in the upper ring area; an oval or circular seal with text in both the upper and lower ring areas; an oval or circular seal with text in the upper, lower, and central ring areas; and a square seal containing text. The definitions of the upper, lower, and central ring areas of a circular seal are the same as those for an oval seal.

[0090] The technical solution provided in this application firstly involves performing seal detection on an object image including a seal to obtain the seal's shape category information, location information, and location information of key points. Next, based on the seal's location information, an original seal image is obtained from the object image, and noise reduction processing is applied to the original seal image to obtain a seal image. Finally, image processing is performed on the seal images using the location information of key points for seals of different shape categories to obtain one or more images to be recognized by text. Text recognition is then performed on these one or more images to obtain the seal's text recognition result. Therefore, a novel seal text recognition method is provided, which can effectively improve the accuracy of seal text recognition and supports text recognition for seals of various styles, colors, and shapes.

[0091] In some optional embodiments of this application, in order to better reduce the impact of noise on the recognition accuracy of seal characters, the original seal image is denoised. When obtaining the seal image, the original seal image can be grayscaled. The grayscale original seal image is then denoised using a pre-trained denoising model to obtain the seal image.

[0092] Specifically, before using the denoising model for noise reduction, the original seal image is first converted to grayscale to reduce noise caused by image color. Furthermore, grayscale conversion improves the robustness of the denoising model; even if the original seal image is not a red seal image, or if the image clarity is insufficient or the printing is uneven due to factors such as insufficient ink, the denoising model can still achieve good noise reduction results. It is worth noting that grayscale conversion enables the seal text recognition method provided in this application embodiment to accurately recognize text from non-red seals, as well as seals with insufficient ink or uneven printing.

[0093] In this embodiment, the denoising model is used to denoise the grayscale image. The denoising model includes, but is not limited to, convolutional neural networks, recurrent neural networks, and long short-term memory networks. Further optionally, to improve the denoising effect, the denoising model may be, for example, a generative adversarial network (GAN). A GAN consists of at least a generator model and a discriminator model. GANs include, but are not limited to, Pixel2Pixel GANs, where Pixel2Pixel refers to a pixel-to-pixel mapping. When the denoising model is a GAN, the generator model in the GAN is used to denoise the original grayscale seal image to obtain the seal image.

[0094] In this embodiment, the stamp image output by the denoising model has removed text that does not belong to the stamp compared to the original stamp image. In other words, the denoising model can reduce the interference of text that does not belong to the stamp. As an example, the training method for the denoising model is as follows: A training dataset is obtained, which includes noisy and clean images of the same sample stamp. For any sample stamp, both the noisy and clean images are converted to grayscale. The grayscale noisy image is input into the generator model of the denoising model to obtain a denoised grayscale image of the sample stamp. The grayscale noisy and denoised grayscale images are input into the discriminator model of the denoising model to obtain a first discrimination result corresponding to the sample stamp. The grayscale noisy and clean images are input into the discriminator model of the denoising model to obtain a second discrimination result corresponding to the sample stamp. The model parameters of the generator model or the discriminator model are adjusted based on the discrimination results of multiple sample stamps.

[0095] Specifically, during the training phase, the stamps used in model training are referred to as sample stamps. For each sample stamp, a noisy stamp image refers to an image containing that sample stamp that is noisy, while a clean stamp image refers to an image containing that sample stamp that is noise-free. Noisy and clean stamp images of multiple sample stamps are collected, and the noisy and clean stamp images of each sample stamp from the multiple sample stamps are added to the training dataset as a single training data set.

[0096] In this embodiment, to enable the denoising model to reduce interference from text that is not part of the seal (i.e., noise), a noisy seal image refers to a seal image that includes text that is not part of the seal; a clean seal image refers to a seal image that does not include text that is not part of the seal. For example, Figure 7a The image of the noisy stamp shown contains the text "Three Hundred Tang Poems," which is not text found on a stamp. Figure 7a The clean seal image shown has the text "Three Hundred Tang Poems" removed.

[0097] In practical applications, we can acquire a noisy stamp image with a red stamp color, convert it to HSV format, where H represents Hue, S represents Saturation, and V represents Value. HSV is a color space created based on the intuitive characteristics of color, also known as the hexagonal pyramid model. Next, we extract the red component from the HSV-formatted noisy stamp image and use this red component as the corresponding clean stamp image. Continuing... Figure 7a For example, suppose Figure 7a The image of the noisy stamp shown is a red stamp, but the text of "Three Hundred Tang Poems" is not red. Figure 7a The clean stamp image shown is extracted in HSV format. Figure 7a The red component is obtained from the noise stamp image shown.

[0098] In practical applications, a training dataset can be obtained based on multiple documents bearing sample stamps, with some documents bearing different sample stamps. Specifically, among the documents bearing sample stamps, some bear the same sample stamp, while others bear different sample stamps. First, for any document bearing a sample stamp, the document is photographed or scanned to obtain an object image including the sample stamp. Then, based on the position of the sample stamp within the object image, an image including the sample stamp is extracted from the object image. Next, a noise detection method, either manually or automatically, is used to determine whether the image including the sample stamp is a noisy stamp image or a clean stamp image. Finally, the noisy stamp image and the clean stamp image of the same sample stamp are added as training data to the training dataset.

[0099] Alternatively, the training dataset can be obtained as follows: the training dataset is obtained based on multiple files stamped with sample seals; multiple clean seal images are stamped onto multiple unstamped files, and multiple new noisy seal images are obtained based on the multiple files with clean seal images; the multiple new noisy seal images and their corresponding clean seal images are used as new training data; the multiple new training data are added to the training dataset to enhance the training dataset.

[0100] Continue with Figure 7a For example, after extracting Figure 7a After showing a clean seal image with a red seal color, Figure 7a The clean stamp image with red color is randomly stamped onto multiple files. Multiple new noisy stamp images are obtained from these files. Text not belonging to the stamp in the new noisy stamp images can be located above or below the stamp text; there are no restrictions. These new noisy stamp images and their corresponding clean stamp images are used as new training data to obtain expanded training data. In this embodiment, during the process of obtaining the training dataset based on multiple files stamped with sample stamps, multiple clean stamp images are obtained. These clean stamp images are then stamped onto multiple unstamped files to obtain multiple new files stamped with sample stamps. The new files stamped with sample stamps are photographed or scanned to obtain new object images. Based on the position of the sample stamp in the new object image, new noisy stamp images are obtained from the new object image. Finally, the clean stamp images and their corresponding new noisy stamp images are added to the training dataset as new training data. Therefore, by using data augmentation to create samples, the training dataset is expanded, which makes the denoising model more robust to the image clarity of the seal. In other words, the denoising effect of the denoising model will not vary greatly due to the different image clarity of the seal.

[0101] In this embodiment, during the training phase, a data augmentation method is used to convert the noisy stamp image input into the denoising model to grayscale, so that the denoising model has good robustness to stamp color, that is, the denoising effect of the denoising model will not vary greatly due to different stamp colors.

[0102] In this embodiment, Pixel2Pixel GAN ​​includes a generator model and a discriminator model. The generator model performs image denoising, and the discriminator model distinguishes between real and fake images. For grayscale noisy stamp images and grayscale clean stamp images of the same stamp, see [link to documentation]. Figure 7bDuring the model training phase, on one hand, the generator model denoises the grayscale noisy stamp image to obtain a denoised grayscale image. The grayscale noisy stamp image and the denoised grayscale image are then input into the discriminator model for real / false discrimination. The discriminator model identifies a false result, indicating that the denoised grayscale image is the output of the generator model. On the other hand, the grayscale noisy stamp image and the grayscale clean stamp image are input into the discriminator model for real / false discrimination. The discriminator model identifies a true result, indicating that the grayscale clean stamp image is not the output of the generator model. It should be noted that... Figure 7b The discriminator model in this example has two components, but this is only for ease of understanding and explanation. In actual applications, the denoising model uses only one discriminator model. In this embodiment, the generator model and discriminator model in the denoising model are trained alternately using adversarial training. For more information on adversarial training, please refer to related technologies. Specifically, when training the generator model, the model parameters of the discriminator model are fixed; when training the discriminator model, the model parameters of the generator model are fixed. If the current training is a generator model, for any sample seal, on the one hand, the grayscale noisy seal image and the denoised grayscale image of the sample seal are input into the discriminator model of the denoising model to obtain the first discrimination result corresponding to the sample seal, and the first loss value between the first discrimination result and the first annotation result represented as a false result is calculated; on the other hand, the grayscale noisy seal image and the grayscale clean seal image of the sample seal are input into the discriminator model of the denoising model to obtain the second discrimination result corresponding to the sample seal, and the second loss value between the second discrimination result and the second annotation result represented as a true result is calculated; the model parameters of the generator model are adjusted according to the first and second loss values ​​corresponding to each sample seal. For example, various numerical calculations such as summation, averaging, or accumulation are performed on the first and second loss values ​​corresponding to each sample seal to obtain the final loss value used for model parameter adjustment, and the model parameters of the generator model are adjusted with the goal of minimizing the loss value.

[0103] If the current training is for a discriminator model, for any sample seal, on one hand, the grayscale noisy seal image and the denoised grayscale image of the sample seal are input into the discriminator model of the denoising model to obtain the first discrimination result corresponding to the sample seal. A first loss value is calculated between the first discrimination result and the first annotation result representing a false result. On the other hand, the grayscale noisy seal image and the grayscale clean seal image of the sample seal are input into the discriminator model of the denoising model to obtain the second discrimination result corresponding to the sample seal. A second loss value is calculated between the second discrimination result and the second annotation result representing a true result. The model parameters of the discriminator model are adjusted based on the first and second loss values ​​corresponding to each sample seal. For example, various numerical calculations such as summation, averaging, or accumulation are performed on the first and second loss values ​​corresponding to each sample seal to obtain the final loss value used for model parameter adjustment. The model parameters of the discriminator model are adjusted with the goal of minimizing the loss value.

[0104] In practical applications, model training is iterated multiple times until the end-of-training condition is met. This end-of-training condition could be that the model has been trained a specified number of times, or that the loss value has reached its minimum.

[0105] To better understand the technical solutions provided in the embodiments of this application, the following is combined with... Figure 7c Here is a specific scenario example:

[0106] In the government sector, there is a large volume of document data that needs to be processed, and the seals on these documents are a crucial piece of information. Recognizing the text on these seals is of immense value to the government sector. In this scenario, a seal text recognition system can be used to recognize the text on square, round, and oval seals.

[0107] Specifically, firstly, the paper version of the stamped document is scanned using a scanner to obtain an image of the object including the stamp, and then the image of the object including the stamp is uploaded as the input image to the stamp text recognition system.

[0108] Next, the seal character recognition system performs seal detection on the input image. The seal detection result includes at least the position information of the seal's detection frame, the position information of the seal's key points, and the shape category of the seal;

[0109] Next, the seal character recognition system obtains the original seal image from the input image based on the position information of the seal's detection box, and then denoises the original seal image to obtain the seal image, which is... Figure 7c Denoising the stamp in the middle;

[0110] Next, the seal character recognition system determines the seal category based on its shape. Seal categories are divided into square seals, round seals, and oval seals.

[0111] If the seal is square, the image is rotated based on the positional information of the key points of the seal to align it correctly. Figure 7c The system performs a seal alignment operation, then performs text recognition on the aligned seal image, and finally outputs the text recognition results.

[0112] If the seal is round (i.e., a circular seal), firstly, the round seal is partitioned and masked, that is, the seal image and its copy are divided into a first sector region excluding the seal's main name and a second sector region including the seal's main name. The first sector region in the seal image and the second sector region in the copy image are then masked.

[0113] Next, the central characters of the round seal are aligned. This involves rotating the masked copy of the seal image to obtain an image for character recognition.

[0114] Next, the circular text on the seal is expanded. That is, the masked seal image is sequentially rotated and transformed into polar coordinates to obtain an image for text recognition, and the masked copy of the seal image is sequentially rotated and transformed into polar coordinates to obtain an image for text recognition.

[0115] Finally, the seal character recognition system performs character recognition on the three images of the round seal and outputs the character recognition results.

[0116] If the seal is an oval seal (i.e., an oval-shaped seal), then the seal image is rotated based on the positional information of the key points of the seal to align the seal image. Figure 7c The operation of straightening the seal in the middle.

[0117] Next, resize the oval seal image to transform it into a circular seal image.

[0118] Next, an elliptical stamp partitioning masking operation is performed. That is, for three identical first circular stamp images, second circular stamp images, and third circular stamp images: the upper ring region in the first circular stamp image is determined, and the image region in the first circular stamp image excluding the upper ring region is masked; the center region in the second circular stamp image is determined, and the image region in the second circular stamp image excluding the center region is masked; the lower ring region in the third circular stamp image is determined, and the image region in the third circular stamp image excluding the lower ring region is masked.

[0119] Next, the circular text unfolding is performed. That is, polar coordinate transformation is performed on the first and third circular stamp images after masking to obtain a separate image for text recognition. In addition, the second circular stamp image after masking is directly used as an image for text recognition.

[0120] Finally, the seal character recognition system performs character recognition on the three images of the oval seal and outputs the character recognition results.

[0121] Figure 8 A flowchart illustrating another seal character recognition method provided in this application embodiment. This method can be executed by a seal character recognition device, which can consist of software and / or hardware, and is generally configured in a cloud server. See also... Figure 8 The method may include the following steps:

[0122] 801. Receive a seal text recognition request sent by the terminal device. The seal text recognition request carries an image of the object including the seal.

[0123] 802. In response to the seal text recognition request, perform seal detection on the object image to obtain the seal detection result, which includes the seal's shape category information, position information, and position information of the seal's key points.

[0124] 803. Obtain the original seal image from the object image based on the seal's position information.

[0125] 804. Denoise the original seal image to obtain the seal image.

[0126] 805. Based on the shape category information and the location information of the key points of the seal, perform image processing on the seal image to obtain at least one image to be recognized by text.

[0127] 806. Perform character recognition on at least one image to be recognized to obtain the character recognition result of the seal.

[0128] 807. Return the character recognition results of the seal to the terminal device.

[0129] For a detailed description of the implementation process and technical effects of each step performed by the cloud server, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0130] This application also provides a method for recognizing characters on a seal, including the following steps:

[0131] S1. Perform seal detection on the object image including the seal to obtain the seal detection result. The seal detection result includes the shape category information, position information and position information of the key points of the seal.

[0132] S2. Obtain the seal image from the object image based on the seal's position information.

[0133] S3. Based on the shape category information and the position information of the key points of the seal, perform image processing on the seal image to obtain at least one image to be recognized by text.

[0134] S4. Perform text recognition on at least one image to be recognized to obtain the text recognition result of the seal.

[0135] This seal character recognition method eliminates the need for noise reduction processing on the seal image obtained from the object image. For details on the implementation of each step in this method, please refer to the relevant descriptions in the foregoing embodiments.

[0136] The technical solution provided in this application firstly involves performing seal detection on an object image including a seal to obtain the seal's shape category information, location information, and location information of key points. Next, the seal image is obtained from the object image based on the seal's location information. Finally, image processing is performed on the seal images using the location information of the key points for seals of different shape categories to obtain one or more images to be recognized by text. Text recognition is then performed on these one or more images to obtain the text recognition result for the seal. Therefore, a novel seal text recognition method is provided, which can effectively improve the accuracy of seal text recognition and supports text recognition for seals of various styles, colors, and shapes.

[0137] Figure 9 This is a schematic diagram of the structure of a seal character recognition device provided in an embodiment of this application. Figure 9 As shown, the device may include:

[0138] The seal detection module 91 is used to perform seal detection on an object image including a seal and obtain the seal detection result, which includes the seal's shape category information, position information, and position information of the seal's key points.

[0139] Image acquisition module 92 is used to acquire the original seal image from the object image based on the seal's position information;

[0140] Image denoising module 93 is used to denoise the original seal image to obtain the seal image;

[0141] Image processing module 94 is used to perform image processing on the seal image based on shape category information and the position information of key points of the seal to obtain at least one image to be recognized by text.

[0142] The text recognition module 95 is used to perform text recognition on at least one image to be recognized, and obtain the text recognition result of the seal.

[0143] Further optionally, when the image processing module 94 performs image processing on the seal image based on the shape category information and the position information of the key points of the seal to obtain at least one image to be recognized, it is specifically used for:

[0144] If the shape category information is square, then the seal image is rotated according to the position information of the key points of the seal to obtain at least one image to be recognized by text; if the shape category information is circular or elliptical, then the seal image is rotated, transformed by polar coordinates and masked according to the position information of the key points of the seal to obtain at least one image to be recognized by text.

[0145] Further optionally, if the shape category information is square, the image processing module 94 performs image rotation processing on the seal image according to the position information of the seal key points to obtain at least one image to be recognized. Specifically, if the shape category information is square and the seal key points include the lower right corner vertex, the lower left corner vertex and the center point of the seal, then according to the position information of the lower right corner vertex and the lower left corner vertex, determine the first angle between the direction of the line connecting the lower right corner vertex and the lower left corner vertex and the horizontal direction; rotate the seal image by the first angle with the center point of the seal as the rotation center to obtain the image to be recognized.

[0146] Optionally, if the shape category information is circular, the image processing module 94 performs image rotation, polar coordinate transformation, and masking on the seal image based on the position information of the seal key points to obtain at least one image to be recognized. Specifically, if the shape category information is circular, and the seal key points include the seal center point, a first circumferential point used to define the first character in the seal's main name, and a second circumferential point used to define the last character in the seal's main name, then based on the position information of the seal center point, the first circumferential point, and the second circumferential point, it determines a first sector area in the seal image that does not include the seal's main name. The process involves determining a second sector-shaped region in a copied seal image that includes the name of the seal's main body, wherein the copied seal image is obtained by copying the seal image; masking the first sector-shaped region in the seal image to obtain a masked seal image, and sequentially performing image rotation and polar coordinate transformation on the masked seal image to obtain a first image to be recognized by text; masking the second sector-shaped region in the copied seal image to obtain a masked copied seal image, and performing image rotation on the masked copied seal image to obtain a second image to be recognized by text; and performing polar coordinate transformation on the second image to be recognized by text to obtain a third image to be recognized by text.

[0147] Optionally, when determining the first sector region in the seal image that does not include the name of the seal's main body and the second sector region in the copied seal image that includes the name of the seal's main body, the image processing module 94 is specifically used to: determine the second included angle between the first connecting line direction and the second connecting line direction based on the position information of the seal's center point, the first circumference point, and the second circumference point; wherein, the first connecting line direction refers to the connecting line direction between the seal's center point and the first circumference point, and the second connecting line direction refers to the connecting line direction between the seal's center point and the second circumference point; determine the sector region in the seal image and the copied seal image with the seal's center point as the vertex and the second included angle as the central angle as the first sector region; and determine the sector region in the copied seal image excluding the first sector region as the second sector region.

[0148] Optionally, if the shape category information is elliptical, the image processing module 94 performs image rotation, polar coordinate transformation, and masking on the seal image based on the position information of the seal key points to obtain at least one image to be recognized. Specifically, this is done as follows: if the shape category information is elliptical, performing image rotation and size adjustment operations sequentially on the seal image based on the position information of the seal key points to obtain a first circular seal image; acquiring a second circular seal image and a third circular seal image obtained by copying the first circular seal image; determining the masking regions in the first, second, and third circular seal images based on the position information of the seal key points in the first circular seal image; performing masking processing on the masking regions in the first, second, and third circular seal images respectively; using the masked second circular seal image as the fourth image to be recognized, and performing polar coordinate transformation on the masked first and third circular seal images respectively to obtain a fifth and a sixth image to be recognized.

[0149] Optionally, when the image processing module 94 determines the masking area in the first circular seal image, the second circular seal image, and the third circular seal image based on the position information of the key points of the seal in the first circular seal image, it specifically performs the following: based on the position information of the third and fourth circumferential points in the first circular seal image, it determines the upper circular area in the first circular seal image that includes the seal's main name, and uses the image area in the first circular seal image other than the upper circular area as the masking area in the first circular seal image, wherein the third circumferential point is used to define the first character in the seal's main name, and the fourth circumferential point is used to define the last character in the seal's main name. Based on the position information of the center point of the seal in the first circular seal image, the central region in the second circular seal image is determined, and the image region in the second circular seal image other than the central region is taken as the masking region in the second circular seal image; wherein, the central region in the second circular seal image does not overlap completely or partially overlaps with the annular region where the seal's main name is located; based on the position information of the third and fourth circumference points in the first circular seal image, the lower annular region in the third circular seal image that does not include the seal's main name is determined, and the image region in the third circular seal image other than the lower annular region is taken as the masking region in the third circular seal image.

[0150] Optionally, when the image denoising module 93 performs denoising processing on the original seal image to obtain the seal image, it is specifically used to: perform grayscale processing on the original seal image; and use a pre-trained denoising model to perform denoising processing on the grayscale original seal image to obtain the seal image.

[0151] Optionally, the above-mentioned device further includes a training module, configured to: acquire a training dataset, wherein the training data in the training dataset includes noisy stamp images and clean stamp images of the same sample stamp; for any sample stamp, perform grayscale processing on the noisy stamp images and clean stamp images of the sample stamp respectively; input the grayscale noisy stamp image of the sample stamp into the generator model of the denoising model to obtain a denoised grayscale image of the sample stamp; input the grayscale noisy stamp image, the grayscale clean stamp image, and the denoised grayscale image of the sample stamp into the discriminator model of the denoising model to obtain the discrimination result of the sample stamp; and adjust the model parameters of the generator model or the discriminator model according to the discrimination results of multiple sample stamps.

[0152] Further optionally, when the training module acquires the training dataset, it is specifically used for: acquiring the training dataset based on multiple files stamped with sample seals; stamping multiple clean seal images onto multiple unstamped files respectively, and acquiring multiple new noisy seal images based on the multiple files stamped with clean seal images; using the multiple new noisy seal images and their corresponding clean seal images as new training data respectively; and adding the multiple new training data to the training dataset to enhance the training dataset.

[0153] Optionally, when the seal detection module performs seal detection on an object image containing a seal, it specifically performs seal detection on the object image containing a seal using a pre-trained seal detection model. The seal detection model is trained based on sample object images and their annotation results. The annotation results of the sample object images include: the shape category information, position information, and position information of the key points of the seal in the sample object image.

[0154] Figure 9 The device shown can perform Figure 2 As shown or Figure 8 The implementation principle and technical effects of the method shown will not be elaborated further. Regarding the above embodiments... Figure 9 The specific ways in which each module and unit of the device shown performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0155] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 205 can be device A; or the execution subject of steps 201 and 202 can be device A, and the execution subject of steps 203 to 205 can be device B; and so on.

[0156] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 201, 202, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0157] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10As shown, the electronic device includes: a memory 11 and a processor 12;

[0158] Memory 11 is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0159] The memory 11 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0160] The processor 12, coupled to the memory 11, is used to execute the computer program in the memory 11 to perform the steps in the above-described seal character recognition method.

[0161] Furthermore, such as Figure 10 As shown, the electronic device also includes other components such as a communication component 13, a display 14, a power supply component 15, and an audio component 16. Figure 10 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 10 The components shown. Additionally... Figure 10 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a desktop computer, laptop computer, smartphone, or IoT (Internet of Things) device, or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a desktop computer, laptop computer, or smartphone, it may include... Figure 10 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., it may be omitted. Figure 10 The component within the dashed box.

[0162] For a detailed description of the implementation process of each action by the processor, please refer to the relevant descriptions in the foregoing method embodiments or device embodiments, which will not be repeated here.

[0163] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by an electronic device in the above method embodiments.

[0164] Accordingly, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enables the processor to perform the steps that can be executed by an electronic device in the above method embodiments.

[0165] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2nd Generation), 3G (3rd Generation), 4G (4th Generation) / LTE (long Term Evolution), 5G (5th Generation), or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0166] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0167] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0168] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (central processing unit, CPU), input / output interfaces, network interfaces, and memory.

[0174] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0175] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0177] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for recognizing characters on a seal, characterized in that, include: Perform seal detection on an object image including a seal to obtain a seal detection result. The seal detection result includes the shape category information, position information, and position information of the key points of the seal. The original seal image is obtained from the object image based on the seal's position information; The original seal image is subjected to noise reduction processing to obtain the seal image; If the shape category information is circular, and the key points of the seal include the center point of the seal, a first circumferential point defining the first character in the seal's main name, and a second circumferential point defining the last character in the seal's main name, then based on the position information of the center point, the first circumferential point, and the second circumferential point, a first sector area excluding the seal's main name in the seal image and a second sector area including the seal's main name in the copied seal image are determined. The copied seal image is obtained by copying the seal image. The first sector-shaped region in the stamp image is masked to obtain a masked stamp image. The masked stamp image is then subjected to image rotation and polar coordinate transformation to obtain a first image to be recognized by text. The second sector-shaped region in the copied stamp image is masked to obtain a masked copied stamp image. The masked copied stamp image is then rotated to obtain a second image to be recognized by text. The second image to be recognized by text is then transformed by polar coordinate transformation to obtain a third image to be recognized by text. This third image is then used as the final image to be recognized by text. Perform text recognition on at least one image to be recognized to obtain the text recognition result of the seal.

2. The method according to claim 1, characterized in that, Also includes: If the shape category information is square, and the key points of the seal include the lower right corner vertex, the lower left corner vertex and the center point of the seal, then based on the position information of the lower right corner vertex and the lower left corner vertex, the first angle between the direction of the line connecting the lower right corner vertex and the lower left corner vertex and the horizontal direction is determined. Using the center point of the seal as the rotation center, rotate the seal image by the first included angle to obtain the image to be recognized.

3. The method according to claim 1, characterized in that, Determining a first sector-shaped region in the seal image that does not include the name of the seal's main body, and determining a second sector-shaped region in the copied seal image that includes the name of the seal's main body, includes: Based on the positional information of the center point of the seal, the first circumferential point, and the second circumferential point, a second included angle between the first connecting line direction and the second connecting line direction is determined; wherein, the first connecting line direction refers to the connecting line direction between the center point of the seal and the first circumferential point, and the second connecting line direction refers to the connecting line direction between the center point of the seal and the second circumferential point. The sector-shaped region in the seal image and the copied seal image, with the center point of the seal as the vertex and the second included angle as the central angle, is defined as the first sector region; The sector region in the copied stamp image, excluding the first sector region, is defined as the second sector region.

4. The method according to claim 1, characterized in that, Also includes: If the shape category information is elliptical, the image rotation and size adjustment operations are sequentially performed on the seal image according to the position information of the key points of the seal to obtain the first circular seal image; Obtain a second circular seal image and a third circular seal image obtained by copying the first circular seal image; Based on the position information of the key points of the seal in the first circular seal image, determine the areas to be masked in the first circular seal image, the second circular seal image, and the third circular seal image; Masking processing is performed on the areas to be masked in the first circular seal image, the second circular seal image, and the third circular seal image, respectively; The masked second circular seal image is used as the fourth image to be recognized by text recognition, and the masked first circular seal image and the masked third circular seal image are respectively transformed by polar coordinates to obtain the fifth image to be recognized by text recognition and the sixth image to be recognized by text recognition.

5. The method according to claim 4, characterized in that, Based on the positional information of key points in the first circular seal image, the regions to be masked in the first, second, and third circular seal images are determined, including: Based on the position information of the third and fourth circumference points in the first circular seal image, the upper ring area including the seal body name in the first circular seal image is determined, and the image area in the first circular seal image other than the upper ring area is taken as the masking area in the first circular seal image. The third circumference point is used to define the first character in the seal body name, and the fourth circumference point is used to define the last character in the seal body name. Based on the position information of the center point of the seal in the first circular seal image, the central region in the second circular seal image is determined, and the image region in the second circular seal image other than the central region is taken as the masking region in the second circular seal image; wherein, the central region in the second circular seal image does not overlap completely or partially overlaps with the annular region where the seal body name is located. Based on the position information of the third and fourth circumference points in the first circular seal image, the lower ring region of the third circular seal image that does not include the seal body name is determined, and the image region in the third circular seal image other than the lower ring region is taken as the masking region in the third circular seal image.

6. The method according to any one of claims 1 to 5, characterized in that, The noise reduction process performed on the original seal image to obtain the seal image includes: The original seal image is converted to grayscale. The original grayscale seal image is denoised using a pre-trained denoising model to obtain the seal image.

7. The method according to claim 6, characterized in that, The training method for the noise reduction model is as follows: Obtain a training dataset, wherein the training data in the training dataset includes noisy stamp images and clean stamp images of the same sample stamp; For any sample stamp, the noisy stamp image and the clean stamp image of the sample stamp are respectively processed into grayscale; The grayscale image of the sample stamp is input into the generator model of the denoising model to obtain the denoised grayscale image of the sample stamp. The grayscale image of the sample stamp and the denoised grayscale image are input into the discriminator model of the denoising model to obtain the first discrimination result corresponding to the sample stamp. The grayscale image of the sample seal (noisy seal) and the grayscale image of the clean seal are input into the discriminator model of the denoising model to obtain the second discrimination result corresponding to the sample seal. The model parameters of the generator model or the discriminator model are adjusted based on the discrimination results of multiple sample seals.

8. The method according to claim 7, characterized in that, Obtain the training dataset, including: The training dataset was obtained from multiple documents stamped with sample seals. Multiple clean stamp images are respectively stamped onto multiple unstamped documents, and multiple new noisy stamp images are obtained based on the multiple documents stamped with the clean stamp images; Multiple new noisy stamp images and their corresponding clean stamp images are used as new training data. Multiple new training data are added to the training dataset to enhance it.

9. A method for recognizing characters on a seal, characterized in that, Applied to cloud servers, the method includes: The terminal device receives a seal text recognition request, which carries an image of an object including a seal. In response to the seal text recognition request, seal detection is performed on the object image to obtain seal detection results, which include the shape category information, position information and position information of the key points of the seal. The original seal image is obtained from the object image based on the seal's position information; The original seal image is subjected to noise reduction processing to obtain the seal image; If the shape category information is circular, and the key points of the seal include the center point of the seal, a first circumferential point defining the first character in the seal's main name, and a second circumferential point defining the last character in the seal's main name, then based on the position information of the center point, the first circumferential point, and the second circumferential point, a first sector area excluding the seal's main name in the seal image and a second sector area including the seal's main name in the copied seal image are determined. The copied seal image is obtained by copying the seal image. The first sector-shaped region in the stamp image is masked to obtain a masked stamp image. The masked stamp image is then subjected to image rotation and polar coordinate transformation to obtain a first image to be recognized by text. The second sector-shaped region in the copied stamp image is masked to obtain a masked copied stamp image. The masked copied stamp image is then rotated to obtain a second image to be recognized by text. The second image to be recognized by text is then transformed by polar coordinate transformation to obtain a third image to be recognized by text. This third image is then used as the final image to be recognized by text. Perform character recognition on at least one image to be recognized to obtain the character recognition result of the seal; The character recognition result of the seal is returned to the terminal device.

10. A method for recognizing characters on a seal, characterized in that, include: Perform seal detection on an object image including a seal to obtain a seal detection result. The seal detection result includes the shape category information, position information, and position information of the key points of the seal. The seal image is obtained from the object image based on the seal's position information; If the shape category information is circular, and the key points of the seal include the center point of the seal, a first circumferential point defining the first character in the seal's main name, and a second circumferential point defining the last character in the seal's main name, then based on the position information of the center point, the first circumferential point, and the second circumferential point, a first sector area excluding the seal's main name in the seal image and a second sector area including the seal's main name in the copied seal image are determined. The copied seal image is obtained by copying the seal image. The first sector-shaped region in the stamp image is masked to obtain a masked stamp image. The masked stamp image is then subjected to image rotation and polar coordinate transformation to obtain a first image to be recognized by text. The second sector-shaped region in the copied stamp image is masked to obtain a masked copied stamp image. The masked copied stamp image is then rotated to obtain a second image to be recognized by text. The second image to be recognized by text is then transformed by polar coordinate transformation to obtain a third image to be recognized by text. This third image is then used as the final image to be recognized by text. Perform text recognition on at least one image to be recognized to obtain the text recognition result of the seal.

11. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 1-10.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-10.

Images