Seal text recognition method, device and equipment

Abstract

The embodiments of this specification provide a seal text recognition method, device and equipment that can be used in the field of artificial intelligence technology. The method includes obtaining a picture to be recognized containing seal text; processing the picture to be recognized using a seal text recognition model to obtain a seal text recognition result corresponding to the picture to be recognized; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized. Using the embodiments of this specification, the seal text detection range can be narrowed, and the arc text detection effect and recognition accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a seal text recognition method, device and equipment. Background Art

[0002] To mitigate contractual risks and the resulting economic losses, it's crucial to verify and identify seal content on various contracts. Currently, banks primarily rely on manual seal verification. However, with the surge in banking business volume, traditional manual verification methods are no longer sufficient. Therefore, using text recognition technology to intelligently and batch-wise identify seals in contract images holds significant value in improving contract audit efficiency.

[0003] Existing technologies primarily rely on the end-to-end scene text recognition algorithm Mask TextSpotter, which uses semantic segmentation to achieve accurate arc text recognition. However, this approach requires high differentiation between text and background in images, making it difficult to accurately locate seal text in contract document images, resulting in insufficient recognition performance.

[0004] Therefore, the industry is in urgent need of a technical solution that can solve the above technical problems. Summary of the Invention

[0005] The embodiments of this specification provide a seal text recognition method, device, and equipment, which can narrow the seal text detection range and improve the arc text detection effect and recognition accuracy.

[0006] The seal text recognition method, device and equipment provided in this specification are implemented in the following ways.

[0007] A seal text recognition method comprises: obtaining a picture to be recognized; wherein the picture to be recognized contains seal text; processing the picture to be recognized using a seal text recognition model to obtain a seal text recognition result corresponding to the picture to be recognized; wherein the seal text recognition model comprises a target seal detection network, a target candidate text box generation network, a target text detection network and a target mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized.

[0008] A seal text recognition device comprises: an acquisition module for acquiring a picture to be recognized; wherein the picture to be recognized contains seal text; a processing module for processing the picture to be recognized using a seal text recognition model to obtain a seal text recognition result corresponding to the picture to be recognized; wherein the seal text recognition model comprises a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized.

[0009] A seal text recognition device includes at least one processor and a memory storing computer-executable instructions. When the processor executes the instructions, the steps of any method embodiment in the embodiments of this specification are implemented.

[0010] A computer-readable storage medium stores computer instructions, which, when executed, implement the steps of any method embodiment in the embodiments of this specification.

[0011] This specification provides a seal text recognition method, device and equipment. In some embodiments, a picture to be recognized can be obtained, wherein the picture to be recognized contains seal text, and the picture to be recognized is processed using a seal text recognition model to obtain a seal text recognition result corresponding to the picture to be recognized; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized. Since the present application integrates the Fast R-CNN seal detection network, the Fast R-CNN text detection network and the Mask network in an end-to-end manner to construct a seal text recognition model, it can not only narrow the seal text detection range and improve the arc text detection effect, but also improve the feature representation capability of the arc text, thereby achieving higher recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings described herein are used to provide a further understanding of this specification, constitute a part of this specification, and do not constitute a limitation of this specification. In the accompanying drawings:

[0013] Figure 1This is a flow chart of an embodiment of a seal text recognition method provided in this specification;

[0014] Figure 2 This is a schematic diagram of the module structure of an embodiment of a seal text recognition device provided in this specification;

[0015] Figure 3 This is a hardware structure block diagram of an embodiment of a seal text recognition server provided in this specification. DETAILED DESCRIPTION

[0016] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments in this specification, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on one or more embodiments in this specification without creative effort shall fall within the scope of protection of the embodiments of this specification.

[0017] The following describes the implementation plan of this specification using a specific application scenario as an example. Figure 1 The flowchart of one embodiment of a seal text recognition method provided in this specification is as follows. Although this specification provides the method operation steps or device structure shown in the following embodiments or figures, the method or device may include more or fewer operation steps or module units based on routine or no creative effort.

[0018] An embodiment provided in this specification can be applied to a client, a server, etc. The client can include a terminal device, such as a smartphone, a tablet computer, etc. The server can include a single computer device, a server cluster consisting of multiple servers, or a server structure of a distributed system, etc.

[0019] It should be noted that the following embodiment description does not limit the technical solutions in other application scenarios that can be expanded based on this specification. Figure 1 As shown, in an embodiment of a seal text recognition method provided in this specification, the method may include the following steps.

[0020] S0: Obtain a picture to be identified; wherein the picture to be identified contains seal text.

[0021] In some implementation scenarios, the image to be identified may include seal text. The image to be identified may be an image including a seal, such as a contract image, bill, expense report, etc.

[0022] In some implementation scenarios, the image to be identified can be obtained from a preset database or memory, wherein the preset database can be Oracle, MySQL, etc.

[0023] S2: Use the seal text recognition model to process the picture to be recognized to obtain the seal text recognition result corresponding to the picture to be recognized; wherein, the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized.

[0024] In some implementation scenarios, after obtaining an image to be identified, a seal text recognition model can be used to process the image to obtain a seal text recognition result corresponding to the image. This recognition result can include the seal text's location and text information. The seal text recognition model can recognize seal text in contract images. Preferentially, it can recognize curved seal text in contract audit scenarios.

[0025] In some implementation scenarios, the seal text recognition model may include a target seal detection network, a target candidate text box generation network, a target text detection network, and a target mask network. The target seal detection network may be used to determine the seal area in the image to be recognized, the target candidate text box generation network may be used to generate text candidate boxes, the target text detection network may be used to determine the target text candidate boxes, and the target mask network may be used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the image to be recognized.

[0026] In some implementation scenarios, the seal text recognition model can be pre-acquired in the following manner: obtaining a training image including seal text; using the training image to train the initial recognition model to obtain a training result; adjusting the parameters of the initial recognition model based on the training result and a preset loss function until the value of the preset loss function is less than a preset value, obtaining the optimal parameters, and using the model corresponding to the optimal parameters as the seal text recognition model. The initial recognition model may include a first seal detection network, a first candidate text box generation network, a first text detection network, and a first Mask network. The seal detection network may be a Fast R-CNN network, the candidate text box generation network may be an RPN (Region Proposal Network), and the text detection network may be a Fast R-CNN network. Fast R-CNN (Fast Region-Convolutional Neural Networks) can normalize the entire image and directly feed it into a deep network, adding candidate box information only when adjacent, and only processing each candidate box in the last few layers. It can also unify category judgment and position refinement into a deep network without the need for additional storage.

[0027] In some implementation scenarios, a training image including seal text can be obtained, and the seal area in the training image can be determined using the first seal detection network. Then, feature extraction is performed on the seal area to obtain image feature information representing the seal area. Then, based on the image feature information and the first candidate text box generation network, multiple text candidate boxes of different sizes are generated. Further, the target text candidate box can be determined based on the regional features corresponding to the text candidate box and the first text detection network. The regional features corresponding to the target text candidate box are input into the first Mask network to obtain the global text instance segmentation map and character segmentation map corresponding to the seal area. Finally, the seal text recognition result can be determined based on the global text instance segmentation map and the character segmentation map, wherein the recognition result includes the position information and text information of the seal text. Further, the loss value can be calculated according to a preset loss function. When the loss value is greater than the preset value, the connection weights of the first seal detection network, the first candidate text box generation network, the first text detection network, and the first Mask network can be adjusted according to the seal text recognition result and the loss function to obtain a second seal detection network, a second candidate text box generation network, a second text detection network, and a second Mask network.

[0028] In some implementation scenarios, after obtaining the second seal detection network, the second candidate text box generation network, the second text detection network and the second Mask network, the second seal detection network, the second candidate text box generation network, the second text detection network and the second Mask network can be used to re-determine the seal text recognition result, and calculate the loss value according to the preset loss function. When the loss value is less than the preset value, the second seal detection network, the second candidate text box generation network, the second text detection network and the second Mask network are used as the target seal detection network, the target candidate text box generation network, the target text detection network and the target Mask network respectively.

[0029] For example, in some implementation scenarios, you can first obtain a trained contract image, send the contract image to the Fast R-CNN1 network for seal detection, distinguish the seal area from the non-seal area, and cut out the part of the seal area in the image. Then, for the cut seal sub-image, use the FPN (Feature Paramial Network) based on ResNet-50 to extract the high-level features and low-level features in the ResNet network, and fuse the high-level features with the low-level features through upsampling to obtain image feature information, realize multi-dimensional feature expression of the seal area, and improve the response capability to text areas of different dimensions. Among them, low-level features refer to the features of the network layer closer to the input layer in the ResNet network. High-level features refer to the features of the network layer closer to the output layer. ResNet can maintain the feature information of each level of the model by retaining it layer by layer, thereby alleviating the gradient disappearance problem of deep networks (such as 50 layers). The FPN network can perform coarse extraction of the seal text area in the seal area and narrow the range.

[0030] In the above implementation scenario, after obtaining the image feature information, the fused image feature information can be passed into the RPN to generate a large number of text candidate boxes of different sizes. ROI Align is used to extract the regional features corresponding to the text candidate boxes, and the extracted regional features are fed into the Fast R-CNN2 network to filter out false positive text candidate boxes that do not contain text (such as those that do not contain text at all). NSM (Non-maximun suppression) is used to filter out redundant text candidate boxes, leaving more compact text candidate boxes, namely the target text candidate boxes. Among them, the network structure of Fast R-CNN1 and Fast R-CNN2 is the same, but the parameters can be adjusted based on the data set during training. The Fast R-CNN2 network can achieve more refined text candidate box regression, reduce the number of text candidate boxes, and improve the computational efficiency of the Mask network in the test phase. ROI Align is a regional feature aggregation method that can be used for feature extraction. In some implementation scenarios, if candidate boxes A and B contain the same text area but have different background positions or sizes, it can be considered that A and B are redundant, and the candidate box with a large background needs to be removed to leave the more compact candidate box.

[0031] In the above implementation scenario, after obtaining the target text candidate frame, the regional features corresponding to the target text candidate frame can be input into the Mask network to generate a global text instance segmentation map and a character segmentation map corresponding to the seal area. Among them, the global text instance segmentation map can be used to determine the position information of the seal text. The outline of the local text instance segmentation map can be understood as a polygon, so that the position information of the polygonal area can be obtained by simply finding the vertices of the polygon. For example, the seal area is a rectangular frame, and the global text instance segmentation map may refer to the polygonal frame of the seal text in this rectangular frame (for arc-shaped text, this polygonal frame is approximately arc-shaped). The character segmentation map is a map corresponding to each character. In some implementation scenarios, the Mask network can be composed of a preset number of convolutional layers and deconvolutional layers. Preferably, in this embodiment, the Mask network includes 4 convolutional layers and one deconvolution layer.

[0032] In the above implementation scenario, after generating the global text instance segmentation map and character segmentation map corresponding to the seal area, the vertex positions of the polygon surrounding the text can be obtained based on the outline of the global text instance segmentation map. Since the Mask network can predict the outline of any polygon of the character area, whether it is horizontal or arc-shaped seal text, the precise position information of the text area can be obtained, and the obtained text area is more compact than the rectangular frame, and the recognition effect of the text is better. Furthermore, the character segmentation map can be decoded into a character sequence through the pixel voting mechanism. Specifically, the background segmentation can be firstly performed. Figure 2Then, the connected areas of the characters are found on the binarized image, the mean of each connected area on the corresponding character segmentation map is calculated, the character corresponding to the maximum mean is assigned to the area, and finally the text sequence corresponding to the current text area is determined from left to right.

[0033] In some implementation scenarios, to improve the recognition accuracy of the obtained seal text recognition model, the weights can be iteratively adjusted based on a preset loss function until the total loss value is less than a preset value. The preset value can be set according to the actual scenario and is not limited in this specification.

[0034] In some implementations, the total loss for each iteration can be calculated by comparing the errors between the predicted polygon outlines, text, and annotation values. This allows the error backpropagation algorithm to adjust the connection weights of each layer to reduce the error and improve the model's recognition accuracy.

[0035] In some implementation scenarios, the preset loss function can include four parts: the loss L of the seal detection network (Fast R-CNN1) rcnn1 , the loss L of the candidate text box generation network (RPN) rpn , the loss L of the text detection network (Fast R-CNN2) rcnn2 And the loss L of the Mask network mask The specific formula can be expressed as:

[0036] L=L rcnn1 +α1L rpn +α2L rcnn2 +α3L mask

[0037] Among them, L represents the value of the preset loss function, α1, α2, and α3 represent L rpn , L rcnn2 and L mask The weight of the Mask network. mask Can include global text instance segmentation loss L global and character segmentation loss L char , that is, L mask =L global +βL char , where β represents the character segmentation loss L char The weight of L global It can be an average binary cross entropy loss function, L char It can be a weighted spatial softmax loss function, specifically, L global and L char It can be determined by the following formula:

[0038]

[0039]

[0040] Among them, N represents the total number of pixels in the global image, y n ∈(0,1) represents the label of the nth pixel, x n Indicates the predicted value of the nth pixel, S(x) represents the sigmoid function, S(x n ) represents x n After the sigmoid transformation, T represents the total number of character categories. The output graph X is a matrix with a dimension of N×T. Y is the label matrix corresponding to X. n.t and Y n,t They represent the probability prediction value and label value of the nth pixel being the tth character, respectively. The weight W is used to balance the character category and background category. n It represents the balance weight of the nth pixel, and n, t, and k represent the sequence numbers.

[0041] In some implementation scenarios, the number of background pixels is N neg , the index of the background category is 0, then the balance weight W of the nth pixel n This can be determined by:

[0042]

[0043] In some implementation scenarios, when the value corresponding to the preset loss function is less than the preset value, training can be stopped, and the corresponding seal detection network, candidate text box generation network, text detection network and Mask network are used as the target seal detection network, target candidate text box generation network, target text detection network and target Mask network respectively, so as to obtain the seal text recognition model.

[0044] In the embodiments of this specification, after the seal text recognition model is obtained, it can be stored accordingly and then directly called when seal recognition is required to obtain the corresponding seal text recognition result.

[0045] In some implementation scenarios, the acquired image to be recognized can be input into a trained seal text recognition model for processing.

[0046] In some implementation scenarios, the use of a seal text recognition model to process the image to be recognized and obtain a seal text recognition result corresponding to the image to be recognized may include: using a target seal detection network to determine the seal area in the image to be recognized; performing feature extraction on the seal area to obtain image feature information representing the seal area; generating multiple text candidate boxes based on the image feature information and a target candidate text box generation network; determining a target text candidate box based on the regional features corresponding to the text candidate boxes and the target text detection network; inputting the regional features corresponding to the target text candidate box into a target mask network to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area; and determining the seal text recognition result based on the global text instance segmentation map and the character segmentation map. The target seal detection network is a Fast R-CNN network that can perform seal detection and distinguish seal areas from non-seal areas.

[0047] In some implementation scenarios, the Fast R-CNN1 network can be used to identify the seal region in the image to be identified, then segment the seal region to obtain a seal sub-image, which can then be subjected to feature extraction. In the embodiments of this specification, by detecting and segmenting the seal region in the contract image, the seal text detection range can be narrowed, reducing interference from background noise, thereby facilitating subsequent accurate location of the seal text.

[0048] In some implementation scenarios, extracting features from the seal area to obtain image feature information representing the seal area may include: extracting features from the seal area using a feature pyramid network to obtain a first image feature and a second image feature; and fusing the first image feature and the second image feature to obtain image feature information representing the seal area. The first image feature may be a high-level feature, and the second image feature may be a low-level feature. Features in the seal area with good expressiveness may be understood as image feature information.

[0049] In some implementation scenarios, for the cut seal sub-image, the FPN based on ResNet-50 can be used to extract high-level features and low-level features in the ResNet network, and then the high-level features are fused with the low-level features through upsampling to obtain image feature information, thereby realizing multi-dimensional feature expression of the seal area and improving the responsiveness to text areas of different dimensions. Among them, low-level features refer to the features of the network layer closer to the input layer in the ResNet network. High-level features refer to the features of the network layer closer to the output layer. Image feature information can include image information such as the shape, size, color, and texture of text and non-text areas. In this way, by constructing and fusing high-level semantic features of different dimensions, the expression ability of text areas of different sizes can be improved.

[0050] In some implementation scenarios, after obtaining image feature information, a network can be used to generate multiple text candidate boxes based on the image feature information and the target candidate text box generation network. For example, the image feature information can be fed into an RPN network to generate a large number of text candidate boxes of different sizes and aspect ratios.

[0051] In some implementation scenarios, after generating multiple text candidate boxes, ROI Align can be used to extract regional features corresponding to the text candidate boxes.

[0052] In some implementation scenarios, after obtaining the regional features corresponding to the text candidate frames, the regional features corresponding to the text candidate frames can be input into the Fast R-CNN network to filter out the text candidate frames that do not contain text, thereby obtaining a first filtered text candidate frame; and the non-maximum suppression method is used to filter out redundant text candidate frames from the first filtered text candidate frames to obtain a target text candidate frame. For example, the extracted regional features can be fed into the Fast R-CNN2 network to filter out false positive text candidate frames that do not contain text (e.g., do not contain text at all), and the NSM (Non-maximun suppression method) is used to filter out redundant text candidate frames, leaving a more compact text candidate frame, i.e., the target text candidate frame.

[0053] In some implementation scenarios, after determining the target text candidate box, the regional features corresponding to the target text candidate box can be input into the target mask network to obtain the global text instance segmentation map and character segmentation map corresponding to the seal area. In some implementation scenarios, the target mask network can include four convolutional layers and one deconvolution layer.

[0054] In some implementation scenarios, determining the seal text recognition result based on the global text instance segmentation map and the character segmentation map may include: determining contour information of the global text instance segmentation map; obtaining vertex position information of a polygon surrounding the seal text based on the contour information; and using all vertex position information as position information of the seal text. The position information may include the vertex positions of the polygon corresponding to the seal text.

[0055] In some implementation scenarios, the determining of the seal text recognition result based on the global text instance segmentation map and the character segmentation map may include: decoding the character segmentation map into a character sequence using a pixel voting mechanism to obtain text information of the seal text. In some implementation scenarios, decoding the character segmentation map into a character sequence using a pixel voting mechanism may be performed by the following steps: first, the background is segmented into Figure 2Then, the connected areas of the characters are found on the binarized image, the mean of each connected area on the corresponding character segmentation map is calculated, and the character corresponding to the maximum mean is assigned to the area. Finally, the text sequence corresponding to the current text area is determined from left to right.

[0056] In some implementation scenarios, after obtaining seal text recognition results, business rules such as keyword matching can be combined to implement batch and intelligent auditing of seals. For example, in some implementation scenarios, after obtaining seal text recognition results, the seal text's preset position information can be used to determine whether the position information of the seal text in the image to be identified is correct, and the seal text content can be used to determine whether the seal text content in the image to be identified is accurate, thereby screening out contracts with seal risks.

[0057] In the embodiments of this specification, based on Mask TextSpotter, the Fast R-CNN seal detection network, the Fast R-CNN text detection network and the Mask network are integrated in an end-to-end manner to construct an improved Mask TextSpotter model (i.e., a seal text recognition model) for seal arc text recognition. This not only narrows the seal text detection range and improves the arc text detection effect, but also improves the feature representation capability of arc text, thereby achieving higher recognition accuracy.

[0058] In this embodiment, seal text is segmented in two-dimensional space using semantics, enabling accurate detection of polygonal text regions within curved text. Based on Mask TextSpotter, a Fast R-CNN network is integrated for seal region detection, narrowing the seal text detection range and reducing background noise interference, thereby improving text detection performance.

[0059] The embodiments of this specification combine the Fast R-CNN seal detection network with MaskTextSpotter in an end-to-end manner, which not only improves the recognition effect of curved text through multi-task complementarity but also reduces the consumption of computing resources.

[0060] Of course, the above is only an exemplary description, and the embodiments of this specification are not limited to the above examples. Technical personnel in the relevant field may make other changes based on the technical essence of this application. However, as long as the functions and effects achieved are the same or similar to those of this application, they should be included in the scope of protection of this application.

[0061] From the above description, it can be seen that the embodiment of the present application can obtain a picture to be identified, wherein the picture to be identified contains seal text, and use the seal text recognition model to process the picture to be identified to obtain the seal text recognition result corresponding to the picture to be identified; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be identified, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain the global text instance segmentation map and character segmentation map corresponding to the seal area in the picture to be identified. Since the present application integrates the Fast R-CNN seal detection network, the Fast R-CNN text detection network and the Mask network in an end-to-end manner to construct a seal text recognition model, it can not only narrow the seal text detection range and improve the arc text detection effect, but also improve the feature representation capability of the arc text, thereby achieving higher recognition accuracy.

[0062] In this specification, the various embodiments of the above method are described in a progressive manner. The same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. For relevant parts, please refer to the partial description of the method embodiment.

[0063] Based on the seal text recognition method described above, one or more embodiments of this specification also provide a seal text recognition device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiment of this specification and is combined with the necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0064] Specifically, Figure 2 This is a schematic diagram of the module structure of an embodiment of a seal text recognition device provided in this specification. Figure 2 As shown, a seal text recognition device provided in this specification may include: an acquisition module 120 and a processing module 122.

[0065] The acquisition module 120 may be used to acquire a picture to be identified; wherein the picture to be identified contains seal text;

[0066] The processing module 122 can be used to process the image to be identified using a seal text recognition model to obtain a seal text recognition result corresponding to the image to be identified; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the image to be identified, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the image to be identified.

[0067] It should be noted that the above-mentioned device may also include other implementation methods according to the description of the method embodiment. The specific implementation methods can refer to the description of the relevant method embodiments and will not be described in detail here.

[0068] This specification also provides an embodiment of a seal text recognition device, comprising a processor and a memory for storing processor executable instructions, wherein the instructions, when executed by the processor, implement the following steps: obtaining a picture to be recognized; wherein the picture to be recognized contains seal text; processing the picture to be recognized using a seal text recognition model to obtain a seal text recognition result corresponding to the picture to be recognized; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the picture to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the picture to be recognized.

[0069] It should be noted that the above-mentioned device may also include other implementations according to the description of the method or apparatus embodiment. Specific implementations can refer to the description of the relevant method embodiment and will not be described in detail here.

[0070] The method embodiments provided in this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 3 This is a hardware structure block diagram of an embodiment of a seal text recognition server provided in this specification. The server can be the seal text recognition device or seal text recognition equipment in the above embodiment. Figure 3As shown, the server 10 may include one or more (only one is shown in the figure) processors 100 (the processor 100 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 200 for storing data, and a transmission module 300 for communication functions. It will be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components shown in the figure may also include other processing hardware, such as a database or multi-level cache, GPU, or other hardware with Figure 3 Different configurations shown.

[0071] The memory 200 can be used to store software programs and modules of application software, such as the program instructions / modules corresponding to the seal text recognition method in the embodiment of this specification. The processor 100 executes various functional applications and data processing by running the software programs and modules stored in the memory 200. The memory 200 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 200 may further include a memory remotely located relative to the processor 100, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0072] The transmission module 300 is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of a computer terminal. In one embodiment, the transmission module 300 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 300 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0073] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The methods or devices described in the above embodiments provided in this specification can implement business logic through computer programs and record them on storage media, and the storage media can be read and executed by computers to achieve the effects of the solutions described in the embodiments of this specification. The storage medium may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical methods. The storage medium may include: devices that use electrical energy to store information, such as various types of memories, such as RAM, ROM, etc.; devices that use magnetic energy to store information, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and USB flash drives; devices that use optical methods to store information, such as CDs or DVDs. Of course, there are other types of readable storage media, such as quantum memories, graphene memories, and so on.

[0075] The above-mentioned seal text recognition method or device embodiment provided in this specification can be implemented by a processor in a computer executing corresponding program instructions, such as using the C++ language of the Windows operating system to implement it on a PC, or on a Linux system, or other systems such as Android or iOS system programming languages to implement it on a smart terminal, as well as processing logic based on a quantum computer, etc.

[0076] It should be noted that the devices, equipment, and systems described above in the specification may also include other implementation methods based on the description of the relevant method embodiments. The specific implementation methods can refer to the description of the corresponding method embodiments and will not be described in detail here.

[0077] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0078] For ease of description, the above devices are described in terms of their functions, divided into various modules. Of course, when implementing one or more of the present inventions, the functions of some modules may be implemented in the same or multiple software and / or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units.

[0079] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices, equipment, and systems according to embodiments of the present invention. It should be understood that it can be implemented by computer program instructions, and these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing a specified function. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] Those skilled in the art will appreciate that one or more embodiments of the present specification may be provided as methods, systems, or computer program products. Thus, one or more embodiments of the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware.

[0081] The foregoing is merely an example of one or more embodiments of this specification and is not intended to limit the one or more embodiments of this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included within the scope of the claims.

Claims

1. A seal text recognition method, characterized in that: include: Obtaining a picture to be identified; wherein the picture to be identified contains seal text; The image to be recognized is processed using a seal text recognition model to obtain a seal text recognition result corresponding to the image to be recognized; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the image to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the image to be recognized, and determine the target text candidate box, including: inputting the area features corresponding to the text candidate box into a Fast R-CNN network to filter out text candidate boxes that do not contain text, to obtain a first filtered text candidate box; using a non-maximum suppression method to filter out redundant text candidate boxes from the first filtered text candidate box, to obtain a target text candidate box; Determine the contour information of the global text instance segmentation map; obtain vertex position information of a polygon surrounding the seal text according to the contour information; and use all vertex position information as position information of the seal text; After obtaining the seal text recognition result, the position information of the seal text in the image to be identified is judged based on the preset position information of the seal text, and the content of the seal text in the image to be identified is judged based on the content of the seal text to check whether the seal risk is detected.

2. The method according to claim 1, characterized in that The process of processing the image to be identified by using the seal text recognition model to obtain the seal text recognition result corresponding to the image to be identified includes: Use the target seal detection network to determine the seal area in the image to be identified; Performing feature extraction on the seal area to obtain image feature information representing the seal area; Generate multiple text candidate frames based on the image feature information and the target candidate text frame generation network; Determine the target text candidate frame based on the regional features corresponding to the text candidate frame and the target text detection network; Input the regional features corresponding to the target text candidate box into the target Mask network to obtain the global text instance segmentation map and character segmentation map corresponding to the seal area; Based on the global text instance segmentation map and the character segmentation map, a seal text recognition result is determined; wherein the recognition result includes position information and text information of the seal text.

3. The method according to claim 2, characterized in that The step of extracting features from the seal area to obtain image feature information representing the seal area includes: Extracting features of the seal area using a feature pyramid network to obtain a first image feature and a second image feature; The first image feature and the second image feature are fused to obtain image feature information representing the seal area.

4. The method according to claim 2, characterized in that The target Mask network includes 4 convolutional layers and one deconvolution layer.

5. The method according to claim 2, characterized in that Determining the seal text recognition result based on the global text instance segmentation map and the character segmentation map includes: The character segmentation map is decoded into a character sequence using a pixel voting mechanism to obtain text information of the seal text.

6. A seal text recognition device, characterized in that: include: An acquisition module is used to acquire a picture to be identified; wherein the picture to be identified contains seal text; A processing module is used to process the image to be recognized using a seal text recognition model to obtain a seal text recognition result corresponding to the image to be recognized; wherein the seal text recognition model includes a target seal detection network, a target candidate text box generation network, a target text detection network and a target Mask network; the target seal detection network is used to determine the seal area in the image to be recognized, the target candidate text box generation network is used to generate a text candidate box, the target text detection network is used to determine the target text candidate box, and the target Mask network is used to obtain a global text instance segmentation map and a character segmentation map corresponding to the seal area in the image to be recognized, and determine the target text candidate box, including: inputting the area features corresponding to the text candidate box into a Fast R-CNN network to filter out text candidate boxes that do not contain text to obtain a first filtered text candidate box; using a non-maximum suppression method to filter out redundant text candidate boxes from the first filtered text candidate box to obtain a target text candidate box; determining contour information of the global text instance segmentation map; obtaining vertex position information of a polygon surrounding the seal text based on the contour information; and using all vertex position information as position information of the seal text; After obtaining the seal text recognition result, the position information of the seal text in the image to be identified is judged based on the preset position information of the seal text, and the content of the seal text in the image to be identified is judged based on the content of the seal text to check whether the seal risk is detected.

7. A seal text recognition device, characterized in that: The method comprises at least one processor and a memory storing computer-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the instructions.

8. A computer-readable storage medium, characterized in that Computer instructions are stored thereon, and when the instructions are executed, the steps of the method according to any one of claims 1 to 5 are implemented.

Images