A deep learning-based grayscale seal coloring method

Abstract

A gray-scale seal coloring method based on deep learning. The present application relates to the technical field of seal identification, and solves the problem that the existing seal identification technology is difficult to separate overlapping gray-scale seals in text. The method comprises the following steps: S1, adopting a target detection network to detect the seal target of a gray-scale document image, and obtaining the position information of the seal in the document image; S2, cutting the seal area according to the position information, taking the seal area as a seal instance image, and unifying the size of the seal instance image; S3, coloring the seal instance image and the gray-scale document image through a coloring network; S4, performing feature fusion on the intermediate layer features of the full-size image coloring network of the instance coloring network through a feature fusion module, and obtaining a global coloring result. The present application can be applied to document seal identification work, and after restoring the color information of the gray-scale seal image, the subsequent seal and background text can be separated according to the color features, and the application prospect is good.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of seal recognition, and particularly relates to a grayscale seal coloring technology based on deep learning. BACKGROUND

[0002] Seals are authentication tools with the highest legal effect, and are widely used in official announcements, contract agreements, license documents and the like. The seals usually contain key information of the issuing authority, and accurate extraction of seal information in a document image is a necessary step in document image recognition. A common seal recognition method is to first extract the seal from the image according to the color characteristics of the seal, separate the seal from the document text, and then analyze and recognize the text information in the seal. This method is suitable for color document images that retain the color characteristics of the seal. Existing seal recognition technologies mostly focus on the recognition and analysis of color seals in color images, but in actual business, there are often a large number of copied and scanned documents, and the text and seals in such documents are usually black, so it is difficult to separate the overlapping seals and text. SUMMARY

[0003] In order to solve the problem that the existing seal recognition technology is difficult to separate the overlapping grayscale seal from the text, the present application provides a grayscale seal coloring method based on deep learning.

[0004] The technical scheme is as follows:

[0005] A grayscale seal coloring method based on deep learning, comprising the following steps:

[0006] S1, performing seal target detection on a grayscale document image by using a target detection network to obtain position information of the seal in the document image;

[0007] S2, cutting the seal region according to the position information to obtain a seal instance image, and unifying the size of the seal instance image;

[0008] S3, coloring the seal instance image and the grayscale document image by using a coloring network;

[0009] S4, performing feature fusion on the intermediate layer features of the full-size image coloring network of the instance coloring network by using a fusion module to obtain a global coloring result.

[0010] Preferably, the target detection network in step S1 is a Faster RCNN network, and the step S1 further comprises:

[0011] S11, extracting image features by using a convolution network module to obtain a feature map;

[0012] S12, extracting a region of interest, i.e. an anchor box of the seal, by using a region candidate module.

[0013] S13, unify the one or more existing regions of interest into a fixed size;

[0014] S14, regress and classify the anchor box through a fully connected layer to obtain position information of the seal in the document image.

[0015] Preferably, the seal instance image size in step S2 is set to 256x256 pixels.

[0016] Preferably, the coloring network in step S3 includes an instance coloring network and a full-size image coloring network, which are respectively used for coloring the seal instance image and the grayscale document image.

[0017] Preferably, the backbone network of the coloring network is a U-net.

[0018] Preferably, the U-net is composed of 10 convolutional layers, wherein convolutional layers 1-4 are used to realize image downsampling operation, reduce the length and width of the feature map, and increase the number of channels; convolutional layers 5-6 increase the receptive field of the network through a hole convolution operation; convolutional layers 7-10 are used to realize upsampling operation, gradually restore the size of the feature map, and reduce the number of channels of the feature map.

[0019] Preferably, the filter size of the 10 convolutional layers is 3x3.

[0020] Preferably, step S4 further comprises:

[0021] S41, input the full-size image feature map, the seal instance image feature map, and the corresponding position information;

[0022] S42, the seal instance image feature map and the full-size image feature map are respectively processed through three layers of convolutional layers to predict the corresponding weight map;

[0023] S43, combine the anchor box information of the seal instance image to adjust the size of the seal instance feature map and the corresponding weight map, and match the original image size through a 0 filling operation;

[0024] S44, combine the weight map to the seal instance feature map and the full-size image feature map to obtain the final feature fusion result.

[0025] The application also provides a computer readable storage medium for storing a computer program, wherein the computer program executes the deep learning-based grayscale seal coloring method as described above.

[0026] The present invention also provides an electronic device, including a processor and a memory, wherein the processor and the memory communicate with each other through a communication bus; the memory is used to store computer programs; the processor is used to execute the computer programs stored in the memory to implement the deep learning-based grayscale stamp coloring method described above.

[0027] Compared with existing technologies, this invention solves the problem of difficulty in separating overlapping grayscale stamps in text, and its specific beneficial effects are as follows:

[0028] 1. This invention can accurately locate the position of the seal in a grayscale document image using Faster RCNN; by coloring the local image of the seal and the original image respectively through two coloring networks, it can effectively combine object-level semantic information; the coloring result of the target local area may be incompatible with the background color, so by fusing the feature maps in the two networks layer by layer through the fusion module, the problem of visual artifacts caused by simple mixing can be effectively avoided.

[0029] 2. By applying the grayscale stamp coloring method provided by this invention, after restoring the color information of the grayscale stamp image, the text of the stamp and background can be separated according to the color features, which is beneficial to the recognition and analysis of document content. This invention has good application prospects in document stamp recognition. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the grayscale stamp coloring method provided by the present invention;

[0031] Figure 2 This is a schematic diagram of the target detection network workflow described in Example 2;

[0032] Figure 3 This is a schematic diagram of the workflow of the coloring network described in Example 4;

[0033] Figure 4 This is a schematic diagram of the workflow of the feature fusion module described in Example 8. Detailed Implementation

[0034] To make the technical solutions of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the following embodiments are only used to better understand the technical solutions of the present invention and should not be construed as limiting the present invention.

[0035] Example 1.

[0036] like Figure 1 As shown, this embodiment provides a grayscale stamp coloring method based on deep learning, including the following steps:

[0037] S1, a target detection network is used to detect a seal target in a gray document image, to obtain position information of the seal in the document image;

[0038] S2, a seal region is cut according to the position information, as a seal instance image, and the size of the seal instance image is unified;

[0039] S3, a coloring network is used to color the seal instance image and the gray document image;

[0040] S4, a feature fusion module is used to fuse the intermediate layer features of the full-size image coloring network of the instance coloring network, to obtain a global coloring result.

[0041] The embodiment solves the problem that the existing seal recognition technology is difficult to separate the overlapped gray seal in the text. After the color information of the gray seal image is restored by using the coloring method provided in the embodiment, the seal and the text in the background can be separated according to the color features, which is beneficial to the recognition and analysis of the document content.

[0042] Embodiment 2.

[0043] In combination with Figure 2 To better understand the embodiment, the embodiment is a further illustration of the embodiment 1, wherein the target detection network in the step S1 is a Faster RCNN network, and the step S1 further includes:

[0044] S11, image features are extracted by a convolution network module, to obtain a feature map;

[0045] S12, a region of interest is extracted by a region candidate module, which is an anchor box of the seal;

[0046] S13, one or more regions of interest are unified to a fixed size;

[0047] S14, regression and class judgment of the anchor box are performed by a full connection layer, to obtain the position information of the seal in the document image.

[0048] The Faster RCNN network is applied in the embodiment, which can accurately locate the position of the seal in the gray document image, to ensure the position accuracy of subsequent coloring and other operations.

[0049] Embodiment 3.

[0050] The embodiment is a further illustration of the embodiment 1, wherein the size of the seal instance image in the step S2 is set to 256x256 pixels.

[0051] Embodiment 4.

[0052] In combination with Figure 3To better understand the present embodiment, the present embodiment is a further illustration of the embodiment 1, wherein the colorization network in step S3 comprises an instance colorization network and a full-size image colorization network, which are respectively used for colorizing the stamp instance image and the grayscale document image. The present embodiment can effectively combine the object-level semantic information by colorizing the stamp local image and the original image through two colorization networks.

[0053] Embodiment 5.

[0054] The present embodiment is a further illustration of the embodiment 4, wherein the backbone network of the colorization network is a U-net.

[0055] Embodiment 6.

[0056] The present embodiment is a further illustration of the embodiment 5, wherein the U-net is composed of 10 convolution layers, wherein convolution layers 1-4 are used to realize image down-sampling operation, reduce the length and width of the feature map, and increase the number of channels; convolution layers 5-6 increase the receptive field of the network through the operation of the hollow convolution; convolution layers 7-10 are used to realize the up-sampling operation, gradually restore the size of the feature map, while reducing the number of channels of the feature map.

[0057] Embodiment 7.

[0058] The present embodiment is a further illustration of the embodiment 6, wherein the filter size of the 10 convolution layers is 3x3.

[0059] Embodiment 8.

[0060] In combination with Figure 4 To better understand the present embodiment, the present embodiment is a further illustration of the embodiment 1, wherein step S4 further comprises:

[0061] S41, input the full-size image feature map, the stamp instance image feature map, and the corresponding position information;

[0062] S42, the stamp instance image feature map and the full-size image feature map are respectively processed through three layers of convolution layers to predict the corresponding weight map;

[0063] S43, in combination with the anchor box information of the stamp instance image, the size of the stamp instance feature map and the corresponding weight map is adjusted, and the original image size is matched through the 0 operation;

[0064] S44, in combination with the weight map, the stamp instance feature map and the full-size image feature map are weighted to obtain the final feature fusion result.

[0065] The embodiment considers that the target local shading result can be incompatible with the background color, and can effectively avoid the problem of visual artifacts generated by simple mixing by layer-by-layer fusion of feature maps in the two networks through the fusion module.

[0066] Embodiment 9.

[0067] The embodiment provides a computer readable storage medium for storing a computer program, the computer program performing the gray stamp shading method based on deep learning as any one of embodiments 1-8.

[0068] Embodiment 10.

[0069] The embodiment provides an electronic device, including a processor and a memory, wherein the processor and the memory complete mutual communication through a communication bus; the memory is used for storing a computer program; the processor is used for executing the computer program stored on the memory, and the gray stamp shading method based on deep learning as any one of embodiments 1-8 is realized.

Claims

1. A deep learning-based grayscale stamp coloring method, characterized by, The method comprises the following steps: S1, using a target detection network to detect the seal target in the gray document image to obtain the position information of the seal in the document image; S2, cutting the seal region according to the position information to obtain a seal instance image, and unifying the size of the seal instance image; S3, coloring the seal instance image and the gray document image through a coloring network; S4, performing feature fusion on the intermediate layer features of the full-size image coloring network of the instance coloring network to obtain a global coloring result; Step S4 further comprises: S41, inputting the full-size image feature map, the seal instance image feature map and the corresponding position information; S42, processing the seal instance image feature map and the full-size image feature map through three convolution layers respectively to predict corresponding weight maps; S43, combining the anchor box information of the seal instance image to adjust the size of the seal instance feature map and the corresponding weight map, and matching the original image size through the 0 operation; S44, combining the weight map to obtain the final feature fusion result through weighting on the seal instance feature map and the full-size image feature map.

2. The deep learning-based grayscale stamp colorization method of claim 1, wherein, The target detection network in step S1 is a Faster RCNN network, and step S1 further comprises: S11, extracting image features through a convolution network module to obtain a feature map; S12, extracting a region of interest, i.e. an anchor box of the seal, through a region candidate module; S13, unifying one or more regions of interest into a fixed size; S14, performing regression and class judgment of the anchor box through a full connection layer to obtain the position information of the seal in the document image. 3.The deep learning-based grayscale stamp colorization method of claim 1, wherein, The size of the seal instance image in step S2 is uniformly set to 256x256 pixels. 4.The deep learning-based grayscale stamp colorization method of claim 1, wherein, The coloring network in step S3 comprises an instance coloring network and a full-size image coloring network, which are respectively used for coloring the seal instance image and the gray document image.

5. The deep learning-based grayscale stamp colorization method of claim 4, wherein, The backbone network of the coloring network is a U-net. 6.The deep learning-based grayscale stamp colorization method of claim 5, wherein, The U-net is composed of 10 convolution layers, wherein convolution layers 1-4 are used to realize image downsampling operation, reduce the length and width of the feature map, and increase the number of channels; convolution layers 5-6 increase the receptive field of the network through the operation of the hollow convolution; convolution layers 7-10 are used to realize the upsampling operation, gradually restore the size of the feature map, and reduce the number of channels of the feature map.

7. The deep learning-based grayscale stamp colorization method of claim 6, wherein, The filter size of the 10 convolution layers is 3x3.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store a computer program, and the computer program executes the gray seal coloring method based on deep learning as claimed in any one of claims 1-7.

9. An electronic device, comprising: The computer readable storage medium is used to store a computer program, and the computer program executes the gray seal coloring method based on deep learning as claimed in any one of claims 1-7.

Images