A method and apparatus for seal image recognition

Abstract

Embodiments of the present application disclose a method and device for seal image recognition. In the embodiments of the present application, a to-be-processed image is obtained, wherein the to-be-processed image includes at least one seal; the to-be-processed image is input into a positioning model to determine the region of each seal; the region of each seal is input into a text recognition model to output text information, wherein the text recognition model has a local attention mechanism; and the text information is input into a retrieval enhancement generation large language model for semantic repair to output target seal text. Through the above method, the accuracy of text recognition of a seal image can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and apparatus for recognizing seal images. Background Technology

[0002] In daily business operations and legal affairs, seals are key visual elements for confirming the legal validity of documents and authenticating identities. These seals include official seals, contract seals, and financial seals. Seal recognition is an important part of automated document processing. However, due to the complex layout, diverse shapes, complex background interference, and problems such as blurriness, damage, overlap, and tilting of seals, seal recognition in documents is quite difficult.

[0003] In existing technologies, the methods of using end-to-end visual multimodal large models and manual review are employed. However, the accuracy of the end-to-end visual multimodal large models is low when processing unstructured, highly rotated, or even distorted seal text, while the manual review method is inefficient, costly, and prone to errors.

[0004] In conclusion, improving the accuracy of text recognition in seal images is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for seal image recognition, which can improve the accuracy of text recognition in seal images.

[0006] In a first aspect, embodiments of the present invention provide a method for recognizing seal images, the method comprising: acquiring an image to be processed, wherein the image to be processed includes at least one seal; inputting the image to be processed into a localization model to determine the location of each seal; inputting the location of each seal into a text recognition model to output text information, wherein the text recognition model has a local attention mechanism; and inputting the text information into a retrieval enhancement generation large language model for semantic repair to output target seal text.

[0007] Optionally, the process of generating the positioning model includes: acquiring multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; training the basic positioning model based on the multiple training stamp data to generate the fine-tuned positioning model.

[0008] Optionally, the text recognition model generation process includes: acquiring multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; training the basic text recognition model based on the multiple training stamp data to generate the fine-tuned text recognition model.

[0009] Optionally, the process of generating the training stamp data includes: acquiring stamp text information, wherein the stamp text information is the text content in the stamp; generating multiple basic stamp images through a stamp data synthesis engine, wherein the multiple basic stamp images include stamps of various shapes; superimposing at least one type of interference on each basic stamp image to generate stamp image data; generating corresponding JSON data based on the stamp image data, wherein the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the frame position, and the rotation angle; and determining the stamp image data and the corresponding JSON data as the training stamp data.

[0010] Optionally, the text recognition model is a Transformer-based optical character recognition model.

[0011] Optionally, the Transformer-based optical character recognition model includes an encoder, a mask prediction head, and a decoder, wherein the mask prediction head includes layer normalization and a linear layer.

[0012] Optionally, the method further includes: updating the parameters of the Transformer-based optical character recognition model according to the total loss, wherein the total loss includes cross-entropy loss and mask calculation error loss.

[0013] Secondly, embodiments of the present invention provide an apparatus for recognizing seal images. The apparatus includes: an acquisition unit for acquiring an image to be processed, wherein the image to be processed includes at least one seal; a determination unit for inputting the image to be processed into a localization model to determine the location of each seal; the determination unit for inputting the location of each seal into a text recognition model to output text information, wherein the text recognition model includes a local attention mechanism; and a generation unit for inputting the text information into a retrieval-enhanced generative language model for semantic repair and outputting target seal text.

[0014] Optionally, during the generation of the positioning model, the acquisition unit is further configured to acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; the device further includes: a training unit, configured to train the basic positioning model based on the multiple training stamp data to generate the fine-tuned positioning model.

[0015] Optionally, during the generation of the text recognition model, the acquisition unit is further configured to acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; the device further includes: a training unit, configured to train the basic text recognition model based on the multiple training stamp data to generate the fine-tuned text recognition model.

[0016] Optionally, during the generation of the training stamp data, the acquisition unit is further configured to: acquire stamp text information, wherein the stamp text information is the text content in the stamp; the generation unit is further configured to: generate multiple basic stamp images through a stamp data synthesis engine, wherein the multiple basic stamp images include stamps of various shapes; the generation unit is further configured to: superimpose at least one type of interference on each basic stamp image to generate stamp image data; the generation unit is further configured to: generate corresponding JSON data based on the stamp image data, wherein the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the frame position, and the rotation angle; the generation unit is further configured to: determine the stamp image data and the corresponding JSON data as the training stamp data.

[0017] Optionally, the text recognition model is a Transformer-based optical character recognition model.

[0018] Optionally, the Transformer-based optical character recognition model includes an encoder, a mask prediction head, and a decoder, wherein the mask prediction head includes layer normalization and a linear layer.

[0019] Optionally, the apparatus further includes an update unit for updating the parameters of the Transformer-based optical character recognition model according to the total loss, wherein the total loss includes cross-entropy loss and mask calculation error loss.

[0020] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect or any one of the possible methods of the first aspect.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any one of the possibilities of the first aspect.

[0022] In this embodiment of the invention, an image to be processed is acquired, wherein the image includes at least one seal; the image is input into a localization model to determine the location of each seal; the location of each seal is input into a text recognition model to output text information, wherein the text recognition model has a local attention mechanism; the text information is input into a retrieval-enhanced generative language model for semantic repair, and the target seal text is output. Through the above method, the text in the seal is determined by the localization model and the text recognition model, and then the text in the seal is repaired by the retrieval-enhanced generative language model, generating a more accurate target seal text, thus improving the accuracy of seal text recognition under different conditions. Attached Figure Description

[0023] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a schematic diagram of a seal in the prior art; Figure 2 This is a flowchart of a method for recognizing a seal image according to an embodiment of the present invention; Figure 3 This is a schematic diagram of an image to be processed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the area where a seal is located in an embodiment of the present invention; Figure 5 This is a flowchart of a method for generating a positioning model according to an embodiment of the present invention; Figure 6 This is a schematic diagram of seal image data in an embodiment of the present invention; Figure 7 This is a flowchart of a text recognition model generation method in an embodiment of the present invention; Figure 8 This is a schematic diagram of a TrOCR architecture in an embodiment of the present invention; Figure 9 This is a schematic diagram of input and output based on a TrOCR architecture in an embodiment of the present invention; Figure 10 This is a schematic diagram illustrating the effect of a local attention mechanism in an embodiment of the present invention; Figure 11 This is a flowchart of a method for generating training seal data in an embodiment of the present invention; Figure 12 This is a schematic diagram of a basic seal image in an embodiment of the present invention; Figure 13 This is a schematic diagram of seal image data in an embodiment of the present invention; Figure 14This is a schematic diagram of a seal image recognition device according to an embodiment of the present invention; Figure 15 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0025] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0026] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0027] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0028] In existing technologies, the complex layout, diverse shapes, complex background interference, and various quality issues of seals make seal recognition in documents quite difficult. Specifically, seal recognition refers to recognizing the text content within the seal. This text content includes Chinese, English, numbers, letters, and may also include text from other languages. The complex layout refers to the text in the seal being arranged in a circular, arc-shaped, or arbitrarily rotated manner, significantly differing from conventional horizontal text. The diverse shapes include various forms such as circles, ovals, and squares, and complex structures such as double-layered seals exist. For example… Figure 1The images show circular (a), elliptical (b), and square (c) seals, but in practice, various shapes and layouts are also used. Background interference includes situations where the seal is placed on text, tables, or patterns, resulting in severe background overlap and occlusion. Quality issues include blurring, damage, overlap, and tilting. Currently, cascaded systems, end-to-end visual multimodal large models, sequence prediction schemes, and general multimodal large models are used. However, these methods have low accuracy when processing unstructured, highly rotated, or even distorted seal text. Manual review is inefficient, costly, and prone to errors. Therefore, improving the accuracy of text recognition in seal images is a problem that needs to be solved.

[0029] In this embodiment of the invention, to solve the above problems, a method for seal image recognition is proposed, specifically as follows: Figure 2 As shown, the method includes: Step S201: Obtain the image to be processed.

[0030] Specifically, the image to be processed includes at least one stamp. The image to be processed can be a document or other image carrying a stamp. The stamp includes text content. Each image to be processed includes one, two, or more stamps. The shape, layout, background, and quality of the stamps vary and are determined according to the actual situation.

[0031] For example, suppose the image to be processed is a contract stamped with a circular seal, for instance. Figure 3 As shown, a circular stamp is affixed to the form of the contract; this is merely an illustrative example.

[0032] Step S202: Input the image to be processed into the positioning model to determine the location of each stamp.

[0033] In one possible implementation, the localization model can be a modified YOLOv11 or another modified model. YOLOv11 is a single-stage target detection algorithm model capable of target localization. The fine-tuned localization model can determine the location of the stamp in the image to be processed. The location of the stamp is a local region in the image to be processed, as shown in the example below. Figure 4 As shown.

[0034] Specifically, the generation process of the positioning model is as follows: Figure 5 As shown, it includes the following: Step S501: Obtain multiple training stamp data.

[0035] Specifically, the training stamp data includes stamp image data and corresponding JSON data.

[0036] Assume that the seal image data is as Figure 6 shown, the Figure 6 content of the seal in is "Tian'a Electronic Technology Co., Ltd. Special Seal for Contracts", each character carries information such as a sequence identifier (which may also be referred to as directory index), and the JSON data includes the text content, the center coordinates of each character in the text content, the directory, the frame position, and the rotation angle. Assume that the JSON data is as follows: "{ "text":"Tian'a Electronic Technology Co., Ltd.", "seal_center": 278, 278 , "text_radius": 184.0, "seal_size": 42, 42 , "seal_type":"standard", "characters": { "char":"Tian", "index": 0, "center_position": { "x": 127.27602385082551, "y": 383.5380642885925 }, "paste_position": { "x": 80, "y": 341 }, "angle_math": 215.0, "rotation_angle_pil": 485.0, "size": { "original_width": 82, "original_height": 79, "compressed_width": 48, "rotated_width": 94, "rotated_height": 85 }, "radius": 184.0 ,…” In the above JSON data, the character "tian" shall also be displayed, while other characters such as "a, dian, zi, ke, ji, you, xian, gong, si" shall not be displayed, which is specifically determined according to the actual situation.

[0037] Step S502: training a basic localization model according to the plurality of training seal data, and generating the fine-tuned localization model.

[0038] In a possible implementation, the plurality of training seal data includes seals of various shapes and with various interferences. The plurality of training seal data is constructed by a seal data synthesis engine, and the specific generation manner will be described in detail later. The generated plurality of training seal data has a large data volume and covers a variety of scenarios, including long-tail scenarios. Training the basic localization model through the plurality of training seal data can improve the performance of the fine-tuned localization model.

[0039] In a possible implementation, the plurality of training seal data further includes real training data in addition to being constructed by the seal data synthesis engine, and the localization model is optimized through mixed training data.

[0040] In the embodiments of the present invention, the seal detection rate in the image is improved and missed detection is reduced through the above steps S501 and S502. The seals include non-standard shape seals, such as elliptical seals, blurred seals and the like.

[0041] Step S203: inputting the region where each of the seals is located into a text recognition model, and outputting text information.

[0042] Specifically, the text recognition model is provided with a Patch Attention mechanism, which enhances the text recognition model's perception capability for local subtle features. The text recognition model is a Transformer-based Optical Character Recognition (TrOCR) model. The TrOCR model directly maps the image features in the region where the seal is located to a character sequence, which is the text information, avoiding error accumulation of the traditional recognition manner. Introducing the Patch Attention mechanism into the text recognition model can enhance the text recognition model's attention to the local details of the seal, and improve the feature extraction capability for low-quality images.

[0043] In one possible implementation, the text recognition model is generated as follows: Figure 7 As shown, it includes the following steps: Step S701: Obtain multiple training stamp data.

[0044] Specifically, the training stamp data includes stamp image data and corresponding JSON data.

[0045] In one possible implementation, the seal image data is as described above. Figure 6 As shown, the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the box position, and the rotation angle, as shown above, and will not be repeated here.

[0046] Step S702: Train the basic text recognition model based on the multiple training stamp data to generate the fine-tuned text recognition model.

[0047] In one possible implementation, the multiple training stamp data includes stamps of various shapes and with various types of interference. The multiple training stamp data is constructed through a stamp data synthesis engine. The specific generation method will be described in detail later. The generated multiple training stamp data has a large amount of data and covers a variety of scenarios, including long-tail scenarios. By training the basic text recognition model with the multiple training stamp data, the performance of the fine-tuned text recognition model can be improved.

[0048] In one possible implementation, the multiple training stamp data, in addition to being constructed through a stamp data synthesis engine, also include real training data, and the text recognition model is optimized by mixing the training data.

[0049] In one possible implementation, the TroOCR model employs a TroOCR architecture, with a lightweight mask predictor appended to the output of the TroOCR encoder. This mask predictor performs binary classification prediction on each Patch Token output by the encoder, generating a prediction mask. The prediction mask is multiplied by the last_hidden_state of the Patch Token to filter out background noise, allowing the TroOCR decoder to focus on the region where the text in the stamp is located.

[0050] The following diagram illustrates the specific structure of the TroOCR architecture. Figure 8As shown, it includes the following: an encoder 801, a mask prediction head 802, and a decoder 803; in the... Figure 8 Based on this, the input and output are described in detail, specifically as follows: Figure 9 The process includes the following steps: The region containing the seal is used as the input image (InputImage). The input image is fed into the Encoder 801, which outputs a local token (Patch Token), which includes a last_hidden_state. The Patch Token is fed into the mask prediction head 802 to generate a predicted mask (Predicted Mask). The Predicted Mask is multiplied by the last_hidden_state of the Patch Token using a local attention mechanism (Patch Attention) to generate a masked hidden state (Masked Hidden State). The Masked Hidden State is fed into the Decoder 803 to generate predicted text (Predicted Text), which is the text in the seal recognized by the fine-tuned text recognition model. The mask prediction head includes a layer normalization layer and a linear layer. The Predicted Mask and the ground truth mask (Ground TruthMask) are used to calculate the mask calculation error loss. The predicted text and the ground truth text (Ground Truth Text) are used to calculate the cross-entropy loss. The total loss is calculated by combining the loss from the mask calculation error and the loss from the mask calculation error. The parameters of the TroOCR model are then updated based on the total loss. Assuming the total loss is denoted as Total_Loss, the specific formula is as follows:

[0051] Among them, the The weighting coefficients are... Representing cross-entropy loss, the The mask calculation error loss is used to supervise the generation of character sequences, and the mask calculation error loss is used to supervise the prediction of text regions at the patch level.

[0052] In one possible implementation, the mask formula corresponding to the mask mapping function of the Patch Attention is a recursive mapping with an output range of [0.9, 1.0]. The recursive mapping formula is as follows: , Among them, the It is the output logit of the mask prediction header for the i-th patch, the This represents the sigmoid function, and the text Patch ( ), at this time the The background patch ( ), at this time the Background suppression is at most 10%; the effect of the Patch Attention is as follows: Figure 10 As shown.

[0053] In one possible implementation, during the mechanical decoding stage of the Decoder, a Beam Search algorithm is used to retain multiple high-probability paths and select the globally optimal solution in combination with a language model. The Beam Search algorithm is a heuristic graph search algorithm commonly used in sequence generation tasks. At each generation step, the K candidate sequences with the highest probability are retained, and the optimal solution is finally selected from these K sequences.

[0054] Step S204: Input the text information into the retrieval enhancement generation large language model and output the target seal text.

[0055] Specifically, the Retrieval-Augmented Generation (RAG) large language model belongs to the Large Language Model (LLM) category. The LLM is a deep learning model based on a Transformer architecture, capable of processing and generating natural language text. It is typically trained on large amounts of text data and possesses the ability to understand and generate language. It is widely used in dialogue systems, text generation, and other natural language processing tasks. The LLM, combined with retrieval-augmented generation technology, utilizes a seal-domain knowledge base to perform semantic-level repair on the text recognition results, significantly improving the accuracy of similar characters and solving the problem of character loss caused by incompleteness or occlusion.

[0056] For example, suppose the text output by the fine-tuned text recognition model is "Qingyuan Water Conservation Studio". After RAG repair, it generates "Qingyuan Water Conservation Studio". This is just an example.

[0057] In one possible implementation, the process of generating the training stamp data is as follows: Figure 11 As shown, it includes the following steps: Step S1101: Obtain the seal text information.

[0058] Specifically, the seal text information refers to the text content within the seal, which is the text to be displayed on the seal, such as "xx Contract Seal", "xx Finance Seal", "xx Studio", etc., depending on the actual situation.

[0059] Step S1102: Generate multiple basic seal images using the seal data synthesis engine.

[0060] Specifically, the multiple basic seal images include seals of various shapes, such as round seals, square seals, etc., as described above. Figure 12 As shown, the Figure 12 In the example, the stamp has a background, but the basic stamp image may not have a background. This is just an example and should be determined according to the actual situation.

[0061] Step S1103: Superimpose at least one type of interference onto each of the basic seal images to generate seal image data.

[0062] Specifically, various complex interference strategies, such as low resolution, blurring, occlusion, watermarking, compression, graffiti, lighting, folding, and copying effects, are introduced into the basic seal image to simulate harsh scenarios in real business operations. The generated seal image data is as follows: Figure 13 As shown, this is only an illustrative example, and the specific details should be determined based on the actual situation.

[0063] Step S1104: Generate corresponding JSON data based on the seal image data.

[0064] Specifically, the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the box position, and the rotation angle.

[0065] Step S1105: Determine the seal image data and the corresponding JSON data as the training seal data.

[0066] In this embodiment of the invention, training seal data is synthesized through a seal data synthesis engine, covering rare characters, traditional characters, and various interference scenarios. The positioning model is fine-tuned and optimized using the training seal data, significantly improving the detection rate of non-standard shaped seals. The text recognition model is also fine-tuned and optimized using the training seal data, improving the text recognition capability of the text recognition model. Furthermore, semantic repair is performed using RAG technology, significantly improving the recognition rate of blurry, incomplete, and low-quality seals.

[0067] In this embodiment of the invention, a device for recognizing seal images is provided, such as... Figure 14As shown, the system specifically includes: an acquisition unit 1401, a determination unit 1402, and a generation unit 1403; wherein, the acquisition unit 1401 is used to acquire an image to be processed, wherein the image to be processed includes at least one seal; the determination unit 1402 is used to input the image to be processed into a localization model to determine the location of each seal; the determination unit 1402 is used to input the location of each seal into a text recognition model to output text information, wherein the text recognition model includes a local attention mechanism; the generation unit 1403 is used to input the text information into a retrieval-enhanced generative language model for semantic repair and output the target seal text.

[0068] Furthermore, during the generation of the positioning model, the acquisition unit is also used to acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; the device further includes: a training unit, used to train the basic positioning model based on the multiple training stamp data to generate the fine-tuned positioning model.

[0069] Furthermore, during the generation of the text recognition model, the acquisition unit is also used to acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; the device further includes: a training unit, used to train the basic text recognition model based on the multiple training stamp data to generate the fine-tuned text recognition model.

[0070] Furthermore, in the process of generating the training stamp data, the acquisition unit is further configured to: acquire stamp text information, wherein the stamp text information is the text content in the stamp; the generation unit is further configured to: generate multiple basic stamp images through a stamp data synthesis engine, wherein the multiple basic stamp images include stamps of various shapes; the generation unit is further configured to: superimpose at least one type of interference on each basic stamp image to generate stamp image data; the generation unit is further configured to: generate corresponding JSON data based on the stamp image data, wherein the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the frame position, and the rotation angle; the generation unit is further configured to: determine the stamp image data and the corresponding JSON data as the training stamp data.

[0071] Furthermore, the text recognition model is a Transformer-based optical character recognition model.

[0072] Furthermore, the Transformer-based optical character recognition model includes an encoder, a mask prediction head, and a decoder, wherein the mask prediction head includes layer normalization and a linear layer.

[0073] Furthermore, the apparatus further includes an update unit for updating the parameters of the Transformer-based optical character recognition model according to the total loss, wherein the total loss includes cross-entropy loss and mask calculation error loss.

[0074] Figure 15 This is a schematic diagram of the structure of the electronic device described in an embodiment of the present invention. For example... Figure 15 As shown, it includes a general computer hardware architecture, which includes at least a processor 1501 and a memory 1502. The processor 1501 and the memory 1502 are connected via a bus 1503. The memory 1502 is adapted to store instructions or programs executable by the processor 1501. The processor 1501 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 1501 executes the instructions stored in the memory 1502 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 1503 connects the above-mentioned components together, and also connects the above-mentioned components to the display controller 1504, the display device, and the input / output (I / O) device 1505. The input / output (I / O) device 1505 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 1505 is connected to the system via an input / output (I / O) controller 1506.

[0075] The instructions stored in memory 1502 are executed by at least one processor 1501 to: acquire an image to be processed; input the image to be processed into a localization model to determine the location of each of the seals; input the location of each of the seals into a text recognition model to output text information; input the text information into a retrieval enhancement generation large language model to output the target seal text.

[0076] Specifically, the electronic device includes: one or more processors 1501 and a memory 1502. Figure 15 Take processor 1501 as an example. Processor 1501 and memory 1502 can be connected via a bus or other means. Figure 15 Taking a bus connection as an example, memory 1502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 1501 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 1502, thereby realizing the above-mentioned method for determining seal image recognition.

[0077] Memory 1502 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 1502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 1502 may optionally include memory remotely located relative to processor 1501, and these remote memories may be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0078] One or more modules are stored in memory 1502 and, when executed by one or more processors 1501, perform the seal image recognition method in any of the above method embodiments.

[0079] As those skilled in the art will recognize, various aspects of the embodiments of the present invention can be implemented as a system, method, or computer program product. Therefore, various aspects of the embodiments of the present invention can take the form of a completely hardware implementation, a completely software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." Furthermore, various aspects of the embodiments of the present invention can take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0080] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, (but not limited to) an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the context of embodiments of the present invention, a computer-readable storage medium can be any tangible medium capable of containing or storing a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0081] Computer-readable signal media may include propagated digital signals having computer-readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and can communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0082] Program code implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0083] Computer program code for performing operations relating to various aspects of embodiments of the present invention can be written in any combination of one or more programming languages, including: object-oriented programming languages ​​such as Java, Smalltalk, C++, etc.; and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0084] The flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to embodiments of the present invention describe various aspects of the embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions (executed via the processor of the computer or other programmable data processing apparatus) create means for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0085] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing apparatus or other means to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing that includes instructions that implement the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0086] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operable steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in flowchart and / or block diagram blocks or blocks.

[0087] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse processing. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A method for recognizing seal images, characterized in that, The method includes: Obtain an image to be processed, wherein the image to be processed includes at least one seal; The image to be processed is input into the positioning model to determine the location of each of the seals; The region containing each of the seals is input into a text recognition model, which outputs text information, wherein... Text recognition models possess a local attention mechanism; The text information is input into a large language model for semantic repair, and the target seal text is output.

2. The method according to claim 1, characterized in that, The process of generating the positioning model includes: Acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; The basic positioning model is trained based on the multiple training stamp data to generate the fine-tuned positioning model.

3. The method according to claim 1, characterized in that, The generation process of the text recognition model includes: Acquire multiple training stamp data, wherein the training stamp data includes stamp image data and corresponding JSON data; The basic text recognition model is trained based on the multiple training stamp data to generate the fine-tuned text recognition model.

4. The method according to claim 2 or 3, characterized in that, The process of generating the training stamp data includes: Obtain seal text information, wherein the seal text information is the text content in the seal; Multiple basic seal images are generated using a seal data synthesis engine, wherein the multiple basic seal images include seals of various shapes; At least one type of interference is superimposed on each of the base seal images to generate seal image data; Generate corresponding JSON data based on the seal image data, wherein the JSON data includes the text content, the center coordinates of each character in the text content, the table of contents, the frame position, and the rotation angle; The seal image data and the corresponding JSON data are determined as the training seal data.

5. The method according to claim 1, characterized in that, The text recognition model is a Transformer-based optical character recognition model.

6. The method according to claim 5, characterized in that, The Transformer-based optical character recognition model includes an encoder, a mask prediction head, and a decoder, wherein the mask prediction head includes layer normalization and a linear layer.

7. The method according to claim 6, characterized in that, The method further includes: The parameters of the Transformer-based optical character recognition model are updated based on the total loss, which includes cross-entropy loss and mask calculation error loss.

8. A device for recognizing seal images, characterized in that, The device includes: An acquisition unit is used to acquire an image to be processed, wherein the image to be processed includes at least one seal; A determining unit is used to input the image to be processed into the positioning model and determine the location of each of the seals; The determining unit is used to input the area where each of the seals is located into the text recognition model and output text information, wherein the text recognition model includes a local attention mechanism; The generation unit is used to input the text information into the retrieval enhancement generation large language model for semantic repair and output the target seal text.

9. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images