Intelligent seal identification method and seal identification system

Through the multimodal seal recognition method of pre-training models and Transformer architecture, the problems of high training costs, limited model accuracy and complex processing processes in seal recognition technology are solved, and efficient and accurate seal recognition is achieved, which improves financial transactions Security and compliance.

CN118918604AInactive Publication Date: 2024-11-08WESHARE TECH SERVICES (SHENZHEN) LTD

Patent Information

Application Number
CN202411044156.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing seal recognition technology has problems such as high training costs, limited model accuracy and complex processing processes, resulting in limited processing speed and risk control capabilities of financial services.

Method used

Using pre-trained models and end-to-end learning frameworks, combining image understanding and text generation, especially using Transformer architecture, multimodal seal recognition is performed, reducing dependence on manual annotation data, integrating seal detection, positioning and text recognition steps, and simplifying Processing process.

Benefits of technology

It significantly reduces training costs, improves seal recognition accuracy and efficiency, enhances the security and compliance of financial transactions, simplifies processing processes, and meets the financial industry's demand for efficient and accurate information verification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918604A_ABST
    Figure CN118918604A_ABST
Patent Text Reader

Abstract

The invention relates to an intelligent seal recognition method and system, the intelligent seal recognition method is used for extracting a recognition result after recognizing a seal in an original document image, and the method comprises the steps: carrying out the image preprocessing of the original document image, and obtaining a to-be-analyzed image; performing seal area detection by taking the to-be-analyzed image as a background, and positioning a target seal area; converting the image content of the target seal area into a seal image; performing multi-mode-based seal identification on the seal image, and extracting an identification result; and performing proofreading and matching on the identification result and known information to complete verification of the identification result. The method has the beneficial effects that the problems of high training cost, limited model precision and complex processing flow in the seal identification process can be solved, that is, the training cost is reduced, the identification precision is improved, and the processing flow is simplified.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of financial information verification, and in particular to an intelligent seal recognition method and a seal recognition system. Background Art

[0002] In the process of financial institutions serving corporate transactions, the authenticity and security of information are of vital importance. As the key to confirming the legal effect and credibility of transactions, the accuracy and efficiency of seal recognition and verification directly affect the speed of financial business processing and risk control capabilities. Existing seal recognition technology mainly relies on traditional OCR methods supplemented by TPS algorithms to process curved or distorted texts. This method still has many shortcomings in actual business.

[0003] Specifically, existing seal recognition technology has three major technical defects: high training cost, limited model accuracy, and complex processing flow. In terms of training cost, the introduction of the TPS algorithm increases the complexity of the model, making the amount of data required for training huge, and requiring extremely high accuracy in data annotation, resulting in high costs for the entire process; in terms of model accuracy, due to the limitations of traditional architectures, even with a large amount of resources invested, the accuracy of existing models in recognizing seal text in complex scenarios is still not ideal; in terms of processing flow, the existing processing flow is generally divided into multiple independent steps. This modular processing method is prone to information loss or errors, affecting the final recognition and verification effect. However, in response to the above technical problems, the industry has not yet proposed a reasonable intelligent seal recognition solution. Summary of the invention

[0004] The technical problem to be solved by the present invention is: how to design an intelligent seal recognition solution that can solve the problems of high training cost, limited model accuracy and complex processing flow in the seal recognition process, that is, to reduce training cost, improve recognition accuracy and simplify processing flow.

[0005] To solve the above problems, the embodiments of the present invention propose an intelligent seal recognition method and a seal recognition system, which can solve the problems of high training cost, limited model accuracy and complex processing flow in the seal recognition process, that is, reduce training cost, improve recognition accuracy and simplify processing flow.

[0006] In terms of reducing training costs, traditional seal recognition technology relies on a large number of manually labeled samples for model training, which is not only time-consuming and labor-intensive, but also costly; the present invention, by introducing advanced pre-training model technology and an end-to-end learning framework, can effectively utilize unlabeled or weakly labeled data for self-optimization training, significantly reducing dependence on expensive manually labeled data, thereby greatly reducing the training cost of the system while ensuring recognition performance.

[0007] In terms of improving recognition accuracy, corporate seals generally contain complex pattern information and subtle text information. Traditional methods are prone to errors when recognizing such complex structures. The present invention integrates image understanding and text generation, especially adopts a high-performance Transformer architecture, which can deeply explore the relationship between seal image features and text semantics. Even in the case of seal deformation, blur or partial occlusion, high-precision recognition of seal text can be achieved, greatly enhancing the security and compliance of financial transactions.

[0008] In terms of simplifying the processing flow, the existing seal recognition process, from the detection of the seal area to the extraction of text information, generally requires multiple independent steps and algorithms, which not only increases the complexity of the system but also may introduce additional errors. The present invention integrates key steps such as seal detection, positioning, and text recognition into a continuous processing flow, reducing the conversion loss in the intermediate links and improving the processing speed of the entire system, thereby ensuring stable recognition in various complex environments and meeting the urgent needs of the financial industry for efficient and accurate information verification.

[0009] In the first aspect, an embodiment of the present invention proposes an intelligent seal recognition method, which is used to identify the seal in the original document image and then extract the recognition result. The method includes: S1, performing image preprocessing on the original document image to obtain the image to be analyzed; S2, performing seal area detection with the image to be analyzed as the background to locate the target seal area; S3, converting the image content of the target seal area into a seal image; S4, performing multimodal seal recognition on the seal image to extract the recognition result; S5, proofreading and matching the recognition result with the known information to complete the verification of the recognition result.

[0010] A further technical solution is that, in S1, image preprocessing is performed on the original document image to obtain an image to be analyzed, including: denoising and image quality enhancement are performed on the original document image to obtain an image to be analyzed, so as to improve the accuracy of subsequent processing.

[0011] Its further technical solution is that, S2, using the image to be analyzed as the background to perform seal area detection and locate the target seal area, includes: directly analyzing the image to be analyzed through a pre-trained image TFM model, and locating the target seal area through a self-attention mechanism.

[0012] A further technical solution is that the S3 converts the image content of the target seal area into a seal image, including: referring to the target seal area, cropping the image to be analyzed, and accurately cropping the seal image to prepare for the subsequent text recognition step.

[0013] A further technical solution is that the image TFM model is an image Transformer model, and the text TFM model is a text Transformer model; S4, performing multimodal seal recognition on the seal image and extracting the recognition result, includes:

[0014] S401, performing TFM feature extraction on the seal image to obtain an image feature sequence Z.

[0015] S402, taking the image feature sequence Z as a condition, mapping it to the text feature space through a mapping function to obtain multiple features, and using the multiple features through the text TFM model to generate a wordpiece-level sequence to obtain a text feature sequence Y1.

[0016] S403, directly integrate the position encoding P into the TFM architecture, so that the text TFM model can learn the spatial layout information of the text and obtain the text feature sequence Y2.

[0017] S404, using a cross entropy loss function to measure the difference between the predicted text feature sequence Y1 and the true label, and the difference between the text feature sequence Y2 and the true label.

[0018] S405, based on the extracted seal text and known company name information, by calculating the edit distance, it is determined whether the two are consistent, and then the recognition result is extracted.

[0019] In the above scheme, the cross entropy loss function can be expressed as the cross entropy loss function L CE express.

[0020] In the above scheme, the real label can be expressed as true express.

[0021] Furthermore, the image feature sequence Z in step S401 is calculated by the formula:

[0022] Z = Transformer image (z1,z2,...,z N )

[0023] Wherein, Z represents an image feature sequence after TFM processing, and the image feature sequence Z includes multiple image content information.

[0024] In one embodiment, the method is used to process an original seal image, which is pre-processed and segmented into N non-overlapping image blocks {P1, P2, ..., P N}, each image block is embedded by the function E:P i →z iConvert to vector representation z i , where z i The formula is:

[0025] z i ∈R d

[0026] Where d is the dimension of the vector, which is aggregated through the self-attention mechanism to capture the global context information. For the specific formulation, see the calculation formula of the image feature sequence Z.

[0027] Furthermore, the text feature sequence Y1 in step S402 is calculated by the formula:

[0028] Y1=Transformer text (G(Z))

[0029] The text feature sequence Y1 represents the final generated wordpiece sequence, that is, the WP sequence, and each wordpiece corresponds to a specific subword or character.

[0030] In one embodiment, the obtained image feature sequence Z is used as a condition and mapped to the text feature space through a mapping function, and the mapping function is:

[0031] G:R N×d →R M×e

[0032] That is, the N*d-dimensional vector in the real number domain is mapped to the M*e-dimensional vector in the real number domain, where M is the length of the potential text feature sequence and e is the text feature dimension. Next, the text TFM model uses multiple features to generate wordpiece-level sequences. For the specific model description, see the calculation formula of the text feature sequence Y1.

[0033] Furthermore, the text feature sequence Y2 in step S403 is calculated by the formula:

[0034] Y2=Transformer text (G(Z)+P)

[0035] The position code P is a dynamic two-dimensional position code that changes with the characteristics of the input sequence, helping the model understand the relationship between characters and effectively recognize them even in curved or irregular arrangements. The difference between Y1 and Y2 is that Y1 represents a wordpiece sequence without a position code, that is, a WP sequence code without a position code.

[0036] In one embodiment, in order to process curved seal text or irregular seal text, the present application does not use the traditional TPS transformation, but directly integrates the position code P into the TFM architecture, so that the model can learn the spatial layout information of the text. Therefore, after adding the position code P, the specific adjusted formula refers to the calculation formula of the text feature sequence Y2.

[0037] Furthermore, the cross entropy loss function L of step S404 is CE , and its calculation formula is:

[0038]

[0039] Among them, M represents the sequence length, that is, the number of sample data, Yi represents the probability value of the i-th element in the predicted sequence Y, that is, the model's predicted probability that the i-th sample belongs to a certain category.

[0040] In one embodiment, during the model training process, a cross entropy loss function L is used CE To measure the predicted text feature sequence Y1 and the true label Y true The difference between the text feature sequence Y2 and the true label Y true The difference between them, see the cross entropy loss function L for the specific formula CE The calculation formula for .

[0041] In a second aspect, an embodiment of the present invention further proposes a seal recognition system, which can realize efficient recognition of seal information and efficient recognition and accurate extraction of seal information, and the seal recognition system is used to execute the method described in the first aspect.

[0042] To summarize, the present application relates to a seal recognition method based on multimodality. Through the end-to-end TFM model system, it not only improves the recognition accuracy and efficiency, but also simplifies the model structure, reduces the dependence on manually annotated data, and realizes the effective recognition of complex seal texts.

[0043] The application of this application scheme in the field of financial information verification has achieved the following technical effects:

[0044] First, improve recognition efficiency and accuracy: Use pre-trained TFM models to reduce reliance on large-scale manually annotated data, improve model training efficiency, enhance the ability to capture complex seal patterns and subtle text features, improve recognition accuracy, reduce the need for manual review, and improve the reliability of automated processing.

[0045] Second, reduce operating costs: By designing a streamlined end-to-end processing flow, integrating steps such as seal detection and text recognition, simplifying the system architecture, reducing hardware resource requirements and software development and maintenance costs, accelerating product iteration cycles, and saving long-term operating expenses.

[0046] Third, enhance system flexibility: adopt a multimodal fusion strategy to enable the model to have strong adaptability to various seal types and text styles, improve system flexibility, and cope with the ever-changing information verification needs in the financial market.

[0047] Fourth, improve user experience: through fast and accurate seal recognition capabilities, accelerate the key document review process, shorten customer waiting time, and improve business processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic flow chart of a smart seal recognition method provided by an embodiment of the present invention.

[0051] Figure 2 Another schematic diagram of the process of the smart seal recognition method provided by the embodiment of the present invention.

[0052] Figure 3 Another flowchart of the smart seal recognition method provided by the embodiment of the present invention.

[0053] Figure 4 A schematic diagram of the framework of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or other features, integers, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0057] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to one or any combination and all possible combinations of the associated listed items, and includes these combinations.

[0058] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0059] Example 1

[0060] See also Figures 1 to 3 , is an intelligent seal recognition method provided by an embodiment of the present invention, the intelligent seal recognition method is used to recognize the seal in the original document image and then extract the recognition result, see Figure 1 as well as Figure 2 The method comprises:

[0061] S1, performing image preprocessing on the original document image to obtain an image to be analyzed.

[0062] S2, performing seal area detection with the image to be analyzed as the background to locate the target seal area.

[0063] S3, converting the image content of the target seal area into a seal image.

[0064] S4, performing multimodal seal recognition on the seal image and extracting a recognition result.

[0065] S5, checking and matching the recognition result with the known information to complete the verification of the recognition result.

[0066] In one embodiment, the S1, performing image preprocessing on the original document image to obtain the image to be analyzed, includes: S11, performing denoising and image quality enhancement processing on the original document image to obtain the image to be analyzed, so as to improve the accuracy of subsequent processing.

[0067] In one embodiment, the S2, performing seal area detection with the image to be analyzed as the background to locate the target seal area, includes: S21, directly analyzing the image to be analyzed through a pre-trained image TFM model, and locating the target seal area through a self-attention mechanism.

[0068] In one embodiment, the step S3, converting the image content of the target seal area into a seal image, includes: S31, cropping the image to be analyzed with reference to the target seal area, and accurately cropping the seal image to prepare for the subsequent text recognition step.

[0069] In one embodiment, see Figure 3 The image TFM model is an image Transformer model, and the text TFM model is a text Transformer model; S4, performing multimodal seal recognition on the seal image and extracting a recognition result, including:

[0070] S401, performing TFM feature extraction on the seal image to obtain an image feature sequence Z.

[0071] S402, taking the image feature sequence Z as a condition, mapping it to the text feature space through a mapping function to obtain multiple features, and using the multiple features through the text TFM model to generate a wordpiece-level sequence to obtain a text feature sequence Y1.

[0072] S403, directly integrate the position encoding P into the TFM architecture, so that the text TFM model can learn the spatial layout information of the text and obtain the text feature sequence Y2.

[0073] S404, using a cross entropy loss function to measure the difference between the predicted text feature sequence Y1 and the true label, and the difference between the text feature sequence Y2 and the true label.

[0074] S405, based on the extracted seal text and known company name information, by calculating the edit distance, it is determined whether the two are consistent, and then the recognition result is extracted.

[0075] Furthermore, the image feature sequence Z in step S401 is calculated by the formula:

[0076] Z = Transformer image(z1,z2,...,z N )

[0077] Wherein, Z represents an image feature sequence after TFM processing, and the image feature sequence Z includes multiple image content information.

[0078] In one embodiment, the method is used to process an original seal image, which is pre-processed and segmented into N non-overlapping image blocks {P1, P2, ..., P N}, each image block is embedded by the function E:P i →z i Convert to vector representation z i , where z i The formula is:

[0079] z i ∈R d

[0080] Where d is the dimension of the vector, which is aggregated through the self-attention mechanism to capture the global context information. For the specific formulation, see the calculation formula of the image feature sequence Z.

[0081] In one embodiment, the text feature sequence Y1 in step S402 is calculated by the formula:

[0082] Y1=Transformer text (G(Z))

[0083] The text feature sequence Y1 represents the final generated wordpiece sequence, that is, the WP sequence, and each wordpiece corresponds to a specific subword or character.

[0084] In one embodiment, the obtained image feature sequence Z is used as a condition and mapped to the text feature space through a mapping function, and the mapping function is:

[0085] G:R N×d →R M×e

[0086] That is, the N*d-dimensional vector in the real number domain is mapped to the M*e-dimensional vector in the real number domain, where M is the length of the potential text feature sequence and e is the text feature dimension. Next, the text TFM model uses multiple features to generate wordpiece-level sequences. For the specific model description, see the calculation formula of the text feature sequence Y1.

[0087] In one embodiment, the text feature sequence Y2 in step S403 is calculated by the formula:

[0088] Y2=Transformertext (G(Z)+P)

[0089] The position code P is a dynamic two-dimensional position code that changes with the characteristics of the input sequence, helping the model understand the relationship between characters and effectively recognize them even in curved or irregular arrangements. The difference between Y1 and Y2 is that Y1 represents a wordpiece sequence without a position code, that is, a WP sequence code without a position code.

[0090] In one embodiment, in order to process curved seal text or irregular seal text, the present application does not use the traditional TPS transformation, but directly integrates the position code P into the TFM architecture, so that the model can learn the spatial layout information of the text. Therefore, after adding the position code P, the specific adjusted formula refers to the calculation formula of the text feature sequence Y2.

[0091] In one embodiment, the cross entropy loss function L in step S404 is CE , and its calculation formula is:

[0092]

[0093] Among them, M represents the sequence length, that is, the number of sample data, Yi represents the probability value of the i-th element in the predicted sequence Y, that is, the model's predicted probability that the i-th sample belongs to a certain category.

[0094] In one embodiment, during the model training process, a cross entropy loss function L is used CE To measure the predicted text feature sequence Y1 and the true label Y true The difference between the text feature sequence Y2 and the true label Y true The difference between them, see the cross entropy loss function L for the specific formula CE The calculation formula for .

[0095] The embodiment of the present invention also provides a seal recognition system, which can realize efficient recognition of seal information and efficient recognition and accurate extraction of seal information, and the seal recognition system is used to execute the method described in the above embodiment. The seal recognition system can solve the problems of high training cost, limited model accuracy and complex processing flow in the seal recognition process, that is, reduce training cost, improve recognition accuracy and simplify processing flow.

[0096] Those skilled in the art can understand the related terms of the above embodiments. The related terms of the intelligent seal recognition method and seal recognition system include:

[0097] OCR (Optical Character Recognition): Optical character recognition is a technology that converts text in an image into editable and searchable data.

[0098] TPS (Thin Plate Spline): Thin plate spline, a geometric transformation method, is often used for image deformation, such as text curvature correction.

[0099] CNN (Convolutional Neural Network): Convolutional neural network, a deep learning model that is good at processing image recognition and classification tasks.

[0100] RNN (Recurrent Neural Network): Recurrent neural network, suitable for sequence data processing, commonly used in natural language processing and time series prediction.

[0101] Transformer: A deep learning model based on the self-attention mechanism, abbreviated as TFM, which is widely used in natural language processing tasks such as text generation and understanding.

[0102] wordpiece: A specific text encoding or word segmentation technology, which belongs to a subword model.

[0103] In summary, in the process of financial institutions serving corporate transactions, the authenticity and security of information are of vital importance. As the key to confirming the legal effect and credibility of transactions, the accuracy and efficiency of seal recognition and verification directly affect the speed and risk control capabilities of financial business processing. Existing seal recognition technology mainly relies on traditional OCR methods and supplemented by TPS algorithms to process curved or distorted texts. This method still has many shortcomings in actual business; specifically, existing seal recognition technology has three major technical defects: high training cost, limited model accuracy, and complex processing flow. The intelligent seal recognition method and seal recognition system described in this application can solve the problems of high training cost, limited model accuracy, and complex processing flow in the seal recognition process, that is, reduce training costs, improve recognition accuracy, and simplify processing flow.

[0104] Example 2

[0105] See also Figure 4 , Figure 4This is a block diagram of an electronic device provided by the present invention. The electronic device can be a terminal or a server, wherein the terminal can be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server can be an independent server or a server cluster composed of servers.

[0106] It includes a processor 111 , a communication interface 112 , a memory 113 , and a communication bus 114 , wherein the processor 111 , the communication interface 112 , and the memory 113 communicate with each other via the communication bus 114 .

[0107] The memory 113 is used to store computer programs.

[0108] In one embodiment of the present invention, the processor 111 is used to implement the method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113 .

[0109] It should be understood that in the embodiment of the present application, the processor 111 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art may use different methods to implement the described functions, but such implementation should not be considered to be beyond the scope of the present invention.

[0111] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0112] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0116] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A smart seal recognition method, characterized in that: The intelligent seal recognition method is used to recognize the seal in the original document image and then extract the recognition result, and the method comprises: S1, performing image preprocessing on the original document image to obtain an image to be analyzed; S2, performing seal area detection with the image to be analyzed as the background to locate the target seal area; S3, converting the image content of the target seal area into a seal image; S4, performing multimodal seal recognition on the seal image and extracting a recognition result; S5, checking and matching the recognition result with the known information to complete the verification of the recognition result.

2. The smart seal recognition method according to claim 1, characterized in that: The step S1, performing image preprocessing on the original document image to obtain the image to be analyzed, includes: The original document image is subjected to denoising and image quality enhancement processing to obtain the image to be analyzed, so as to improve the accuracy of subsequent processing.

3. The smart seal recognition method according to claim 2, characterized in that: The step S2, performing seal area detection with the image to be analyzed as the background to locate the target seal area, includes: Through the pre-trained image TFM model, the image to be analyzed is directly analyzed, and the target seal area is located through the self-attention mechanism.

4. The smart seal recognition method according to claim 3, characterized in that: The step S3, converting the image content of the target seal area into a seal image, comprises: Referring to the target seal area, the image to be analyzed is cropped to accurately crop the seal image, so as to prepare for the subsequent text recognition step.

5. The smart seal recognition method according to claim 4, characterized in that: The step S4, performing multi-modal seal recognition on the seal image and extracting a recognition result, includes: S401, performing TFM feature extraction on the seal image to obtain an image feature sequence Z; S402, taking the image feature sequence Z as a condition, mapping it to the text feature space through a mapping function to obtain multiple features, and using the multiple features to generate a wordpiece-level sequence through a text TFM model to obtain a text feature sequence Y1; S403, directly integrating the position code P into the TFM architecture, so that the text TFM model can learn the spatial layout information of the text and obtain the text feature sequence Y2; S404, using a cross entropy loss function to measure the difference between the predicted text feature sequence Y1 and the true label, and the difference between the text feature sequence Y2 and the true label; S405, based on the extracted seal text and known company name information, by calculating the edit distance, it is determined whether the two are consistent, and then the recognition result is extracted.

6. The smart seal recognition method according to claim 5, characterized in that: The image feature sequence Z in step S401 is calculated as follows: Z=Transformer image (z1,z2,...,z N ) Wherein, Z represents an image feature sequence after TFM processing, and the image feature sequence Z includes multiple image content information.

7. The smart seal recognition method according to claim 5, characterized in that: The text feature sequence Y1 in step S402 is calculated as follows: Y1=Transformer text (G(Z)) The text feature sequence Y1 represents the final generated wordpiece sequence, that is, the WP sequence, and each wordpiece corresponds to a specific subword or character.

8. The smart seal recognition method according to claim 5, characterized in that: The text feature sequence Y2 in step S403 is calculated as follows: Y2=Transformer text (G(Z)+P) The position code P is a dynamic two-dimensional position code, which changes with the characteristics of the input sequence, helping the model to understand the relationship between characters and effectively recognize them even in curved or irregular arrangements.

9. The smart seal recognition method according to claim 5, characterized in that: The cross entropy loss function L of step S404 CE , and its calculation formula is: Among them, M represents the sequence length, that is, the number of sample data, Yi represents the probability value of the i-th element in the predicted sequence Y, that is, the model's predicted probability that the i-th sample belongs to a certain category.

10. A seal recognition system, characterized in that: The seal recognition system is used to execute the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Seal extraction and character recognition method and device

    CN112329756A

  • Intelligent seal identification method

    CN113554021A

  • Text document-based character recognition method and device, equipment and storage medium

    CN113569833A

  • Seal identification method and system based on transformer structure

    CN117496522A

Cited By

  • Seal identification method, system and equipment based on generative seal identification model and storage medium

    CN120163970A

  • Seal recognition methods, systems, devices, and storage media based on generative seal recognition models.

    CN120163970B

  • Multi-modal large model-based seal detection and identification method and system

    CN120913186A