Seal character recognition method and system

Abstract

The present invention provides a seal text recognition method and system. The recognition method comprises: detecting a seal to obtain a seal slice image, sequentially performing classification prediction, denoising, text detection, and text recognition steps, and then outputting a text recognition result; performing feature extraction on the detected seal slice image to obtain a feature vector, performing similarity calculation with a seal database, and outputting TopN matching result information; finally, performing similarity calculation between the TopN matching results and the text recognition result, and outputting final seal text information based on logical judgment. The recognition method of the present invention proposes a set of seal text recognition processes, combining image feature template matching with text recognition to achieve accurate seal text recognition.

Description

Technical Field

[0001] The present invention relates to the field of seal character processing, and in particular to a seal character recognition method and system. Background Art

[0002] Seals, as tokens of authenticity, hold a crucial position in government departments, organizations, and enterprises at all levels. Seals are particularly frequently used in financial institutions, banks, and government agencies, and the resulting documents bearing seals are both diverse and numerous. Consequently, the manpower required to identify and process these documents is increasing, significantly reducing the efficiency of relevant personnel. With the advancement of image processing technology, technologies such as optical character recognition (OCR) have also made significant progress. OCR can analyze and identify image files of text documents, extracting text and layout information. Specifically, it recognizes the text within an image and returns it as text. Furthermore, with the increasing prevalence of electronic and paperless office processes, the digitization of seals is also becoming a trend. Using image processing and optical character recognition technologies to detect and recognize the text within Chinese seals, and using machines to rapidly detect and classify the seal content, can effectively improve the efficiency and accuracy of document classification, save manpower, and have significant application value.

[0003] The difficulties in seal text recognition in the existing technology are: (1) the background of the seal in the document is diverse, and these noises cause great interference to the text recognition on the seal; (2) the seal styles are diverse, and it is difficult to recognize the text of seals of different styles.

[0004] In view of this, the present invention is proposed. Summary of the Invention

[0005] In view of this, the present invention discloses a seal text recognition method and system. The recognition method proposes a seal text recognition process, which combines image feature template matching with text recognition to achieve accurate seal text recognition.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention discloses a method for recognizing seal characters, the method comprising:

[0008] The seal slice image obtained by seal detection is subjected to classification prediction, denoising, text detection and text recognition steps in sequence, and then the text recognition result is output;

[0009] The detected seal slice image is subjected to feature extraction to obtain a feature vector, and the similarity is calculated with the seal base database to output the matching result information TopN.

[0010] In a second aspect, the present invention discloses a seal character recognition system, comprising:

[0011] Text recognition module: used to detect the seal to obtain the seal slice image, perform classification prediction, denoising, text detection and text recognition steps in sequence, and then output the text recognition result;

[0012] Feature matching module: It is used to extract features from the detected seal slice image to obtain a feature vector, calculate the similarity with the seal base database, and output the matching result information TopN.

[0013] In a third aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the seal character recognition method as described in the first aspect.

[0014] In a fourth aspect, the present invention discloses a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the seal character recognition method as described in the first aspect are implemented.

[0015] The present invention combines feature matching with seal text recognition to obtain seal text information. The text recognition function is optimized in the seal scenario: first, it accommodates seal diversity, and the target detection and seal text detection methods are compatible with seal targets of different colors and shapes; second, for complex seal backgrounds, background noise removal processing is performed during text recognition to eliminate noise interference; at the same time, considering the advantages of image features, the seal matching method is used to jointly realize seal text recognition. Moreover, the application of the method of the present invention is not limited to seals of a specific style and color, and it has good robustness against complex background noise interference. Therefore, it can adapt to the extraction of seal text information of different styles, and thoroughly improves the efficiency and intelligence level of complex seal information processing.

[0016] The method of the present invention is actually divided into two branches to achieve the ultimate goal together.

[0017] Branch 1 is the text recognition process. (1) Text recognition accommodates the diversity of seal styles, classifies the input seals, and then uses different text detection methods to accurately locate the coordinates of the text area, providing a comprehensive text information area for text recognition. (2) To address the interference of seal background noise on text recognition, the seal image is denoised before seal text detection, which provides a guarantee for text recognition accuracy.

[0018] The second branch is seal feature matching. Considering the high degree of differentiation inherent in image features, this method employs image feature matching to output the highest-probability match. A convolutional neural network model extracts the feature vectors of the seal slice image and compares them with the corresponding feature vectors in the underlying database. The distance between the image and database feature vectors is calculated, and the minimum distance method is used to determine the category. The two branches output the text recognition and matching results, respectively, and the final text information is obtained through logical analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0020] Figure 1 A schematic flow chart of a seal character recognition method provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the structure of a seal character recognition system provided by an embodiment of the present invention;

[0022] Figure 3 A schematic structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0024] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0025] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0026] The present invention discloses a method for recognizing seal characters, the method comprising:

[0027] The seal slice image obtained by seal detection is subjected to classification prediction, denoising, text detection and text recognition steps in sequence, and then the text recognition result is output;

[0028] The detected seal slice image is subjected to feature extraction to obtain a feature vector, and the similarity is calculated with the seal base database to output the matching result information TopN.

[0029] Figure 1 This is a flow chart of the seal character recognition method disclosed in the embodiment of the present invention, referring to Figure 1 As shown, the method includes the following steps:

[0030] Step 1 is seal detection. First, the seal in the image is detected. The present invention uses yolov4 to perform seal target detection, obtains the seal slice image, and then proceeds to the next step of seal classification.

[0031] Step 2 is seal classification. The seal images obtained in step 1 are classified. Because the text strips on square and round (elliptical) seals often have different orientations, different methods are used to accurately locate the seal text information. Therefore, seal classification is first necessary. This paper uses the VGG16 binary classification network model to perform seal classification prediction.

[0032] Step 3 involves seal denoising. The seal slice image obtained from seal detection may contain background noise, which can significantly interfere with text recognition. Therefore, we pre-process the image using the pix2pix method to denoise it, preparing it for subsequent text detection and recognition.

[0033] Step 4 involves seal character detection. Seal character detection is divided into square seal character detection and circular (elliptical) seal character detection. Different methods are used to predict the position of the character strips for different seal types. This method uses DBNet to detect square seal characters and DRRG to segment circular (elliptical) seal characters. The resulting text box information is then used for the next step of text recognition.

[0034] Step 5 is text recognition. The present invention uses crnn to perform text recognition and outputs the text recognition results.

[0035] Step 6 involves feature extraction. First, a feature extraction model is trained. The basic network structure used for training is a twin network. Triple loss is used to adjust training parameters to increase the intra-class spacing of samples and achieve better classification accuracy. Secondly, feature matching typically involves establishing a standard feature base beforehand. A balanced sample of seal images is prepared, and the seal feature base is generated using a feature extraction model. When storing the feature model, each vector corresponds to the label of the corresponding seal.

[0036] Step 7 is feature matching. The similarity between the current vector and the feature vector in the underlying database is calculated, and the output results are sorted from largest to smallest, and the TopN1 scores and set label information are output.

[0037] Step 8 is the logic calculation. Figure 1 The logical calculation process for the two-branch output results is detailed in Figure 1. First, the TopN seal matching result information is determined. If a TopN score is greater than Thr1, the similarity calculation is performed between the tag list that meets the threshold and the text recognition result. Otherwise, the text recognition result is output. Once the conditions are met, the minimum edit distance is calculated between the n tags that meet the threshold and the text recognition result, and then sorted to obtain n similarity scores CTopN. If a CTopN score is greater than Thr2, the tag information of CTop1 in CTopN is output; otherwise, the text recognition result is output.

[0038] Figure 2 The present invention discloses a seal character recognition system, which includes:

[0039] The text recognition module 101 is used to detect the seal to obtain a seal slice image, perform classification prediction, denoising, text detection and text recognition steps in sequence, and then output the text recognition result;

[0040] Feature matching module 102: extracts features from the detected seal slice image to obtain a feature vector, calculates similarity with the seal base database, and outputs matching result information TopN.

[0041] The recognition system is mainly composed of the above two modules, so as to achieve the purpose of accurately identifying seal text by combining image feature template matching and text recognition.

[0042] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0043] Figure 3 This is a schematic diagram of the structure of a computer device disclosed in the present invention. Figure 3 As shown, the computer device includes: an input device 63, an output device 64, a memory 62 and a processor 61; the memory 62 is used to store one or more programs; when the one or more programs are executed by the one or more processors 61, the one or more processors 61 implement an identification method provided in the above embodiment; wherein the input device 63, the output device 64, the memory 62 and the processor 61 can be connected by a bus or other means, Figure 3 The bus connection is taken as an example.

[0044] The memory 62 is a readable and writable storage medium of a computing device, which can be used to store software programs, computer executable programs, such as program instructions corresponding to an identification method described in an embodiment of the present application; the memory 62 may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 62 may further include a memory remotely located relative to the processor 61, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The input device 63 may be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device; the output device 64 may include a display device such as a display screen.

[0046] The processor 61 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 62.

[0047] The computer device provided above can be used to execute an identification method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0048] The present application embodiment also provides a storage medium containing computer executable instructions, which are used to perform a method of identification provided by the above embodiment when executed by a computer processor, and the storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, the second computer system being connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected by a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0049] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present application, whose computer-executable instructions are not limited to an identification method described in the above embodiment, can also execute related operations in an identification method provided in any embodiment of the present application.

[0050] Finally, it should be noted that although this specification contains many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of what is claimed, but are primarily intended to describe the features of specific embodiments of a particular invention. Certain features described in multiple embodiments within this specification may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may function in certain combinations as described above and may even be initially claimed as such, one or more features from a claimed combination may in some cases be removed from the combination, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0051] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0052] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential sequence to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0053] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for recognizing seal characters, characterized in that: The steps include: The seal slice image obtained by seal detection is subjected to classification prediction, denoising, text detection and text recognition steps in sequence, and then the text recognition result is output; Perform feature extraction on the detected seal slice image to obtain a feature vector, calculate the similarity with the seal base database, and output the matching result information TopN; The method for calculating similarity with the seal base database includes: if there is a TopN score greater than a threshold Thr1, then calculating similarity between the label list that meets the threshold and the text recognition result, otherwise outputting the text recognition result; Calculate the minimum edit distance between the n tags that meet the threshold and the text recognition result, sort them to obtain n similarity scores CTopN, and if there is a CTopN score greater than the threshold Thr2, output the tag information of CTop1 in CTopN, otherwise output the text recognition result.

2. The seal character recognition method according to claim 1, wherein: During the feature extraction process, the feature extraction model is first trained. The basic network structure of the training adopts the twin network, and triple loss is used to adjust the training parameters.

3. The seal character recognition method according to claim 1, wherein: The classification prediction is performed using the VGG16 binary classification network model.

4. The seal character recognition method according to claim 1, wherein: The denoising method uses the pix2pix method.

5. The seal character recognition method according to any one of claims 1 to 4, characterized in that: Use dbnet to implement text detection of square seals, and use drrg to implement text segmentation detection of round seals.

6. A seal character recognition system, using the seal character recognition method according to any one of claims 1 to 5, characterized in that: include: Text recognition module: used to detect the seal to obtain the seal slice image, perform classification prediction, denoising, text detection and text recognition steps in sequence, and then output the text recognition result; Feature matching module: It is used to extract features from the detected seal slice image to obtain a feature vector, calculate the similarity with the seal base database, and output the matching result information TopN.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the seal character recognition method according to any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the seal character recognition method as described in any one of claims 1 to 5 are implemented.

Images