Seal identification method based on twin network and space transformation

Abstract

The application provides a seal identification method based on a twin network and a space transformation, and the seal data is uneven in angle, so that preliminary rotation correction normalization is carried out, and a large-angle sample can cause model training to be not convergent, so that the seal to be identified is preliminarily corrected through pretreatment, and the problem of uneven samples is solved. Based on the twin network idea, a similarity measurement network is self-built, and a space transformation network is introduced to further correct the rotation of the seal data, the seal data is further corrected through the STN network, the affine transformation suitable for seal classification is obtained through the network, and the identification accuracy is improved on this basis. The method has feasibility and effectiveness for the identification of the special data of the seal. The application can avoid the situation that the algorithm is invalid when new types of seal data are introduced, can achieve the effect of one-time training and permanent use, and reduces the complexity of training the network to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of image recognition, specifically to a seal recognition method based on Siamese networks and spatial transformation, which can quickly and accurately identify electronic seal images. Background Technology

[0002] With the digitization of archives, seals and imprints, as tools used as identity credentials and symbols of legal person rights, are a key focus of digital management. In seal classification, different types of seals often exhibit significant similarities in appearance, such as identical external shapes and central patterns. However, within the same category, there are considerable differences, such as pattern distortion and incomplete ink application. Early seal classification relied primarily on manually designed feature information, such as local and global character features, stroke topology, and character edge information. While simple, the classification results were not ideal; therefore, extracting robust seal features remains a challenging task.

[0003] Seal classification algorithms can be categorized into traditional methods and convolutional neural network-based methods. K Ude and Y Nakamura proposed a statistical decision-making method based on local and global features of the seal for template matching. This method extracts local and global features, such as the coordinates of outer contour boundary points, character line width, stroke direction, and mean line width, after registration via translation and rotation. However, this method is susceptible to noise. Fan and Tsai proposed a classification method based on the relative stability of the seal stroke topology, extracting the thinned stroke skeleton of the seal character and using skeleton registration for matching decisions, but this method is also easily affected by noise. Horiuchi segmented the seal image into equally spaced small regions and used the pixels within as feature vectors, employing an improved recurrent convolution that only considers positive differences to calculate the registration deflection angle; however, this algorithm is highly sensitive to seal tilt. Lee Kim introduced a stroke feature map, combining the relationship map with geometric position; however, this method struggles to identify and verify seals when their central skeletons are similar but their stroke thickness differs. R. Haruki et al. proposed an algorithm for extracting the edge features of circular seals that reduces the impact of tilted stamping. Jia Hong-yu

[0004] Ge Zheng proposed an improved seal verification method based on stroke edge matching and image difference analysis. With the development of Convolutional Neural Networks (CNNs) in digital image recognition, related technologies are expected to provide better feature extraction and classification methods for seal classification algorithms. However, there are almost no specific CNN classification algorithms for seals; industry classification algorithms are usually validated on general datasets such as VGG and ResNet. However, seal datasets are characterized by a large number of sample types, a small number of samples of the same type, and small inter-class differences, making it difficult for classification algorithms on general datasets to achieve good results. Furthermore, in practical seal recognition tasks, the trained model cannot effectively identify new seal sample types, requiring a new round of training for the new samples. Summary of the Invention

[0005] To effectively address the aforementioned problems and address the special characteristics of seals, this invention proposes a seal recognition method based on Siamese networks and spatial transformation. This method can quickly and accurately identify electronic seal images. First, a gradient histogram-based seal rotation correction method is used to perform preliminary rotation correction on the seal image to be identified. Then, the loss function is modified on the original Siamese network structure, introducing a Spatial Transformation Network (STN) and constructing a similarity measurement network. After preliminary normalization of the seal samples, the STN network is used for further rotation correction of the seal data. The network obtains an affine transformation suitable for seal classification, ultimately completing the identification of the seal type and improving the recognition accuracy.

[0006] The technical solution steps of this invention are as follows:

[0007] Step 1: Select a standard seal image from the reserved seal template set, denoted as O, and map the standard seal image O into a gradient histogram to obtain the horizontal coordinate position Xo of the main peak of the histogram.

[0008] Step 2: Filter the image to be corrected (median filtering) to obtain the image D to be corrected. Map the image D to be corrected into a gradient histogram to obtain the position Xd of the main peak of the histogram.

[0009] Step 3: Obtain the rotation angle A = offset * b by using the offset of the main peak of the histogram offset = XoXd, where b is the angle difference, and finally obtain the corrected image of the seal to be identified, D_j.

[0010] Step 4: Build a basic twin network N, which consists of a series of convolutional layers with the same structure in two paths.

[0011] Step 5: Add a spatial transformation layer at the beginning of the Siamese network N input and modify the loss function, changing the adversarial loss function to binary cross entropy, thus obtaining the spatial transformation network. The input of the spatial transformation network is 105*105 and the output is 105*105.

[0012] Step 6: Train the spatial transformation network.

[0013] Step 7: Input the corrected seal image Dj obtained in Step 3 into the trained spatial transformation network to finally obtain the seal recognition result.

[0014] The beneficial effects of this invention are:

[0015] This invention addresses the issue of uneven angles in seal data by proposing a gradient histogram-based correction algorithm. This algorithm performs initial rotation correction and normalization, as large-angle samples can lead to model training non-convergence. Preprocessing addresses this unevenness by providing initial correction for the seals to be identified. Furthermore, based on the Siamese network concept, a self-built similarity measurement network is constructed, and a Spatial Transformation Network (STN) is introduced for further rotation correction of the seal data. Finally, the similarity of input seal pairs is measured, and the seals are classified. This invention demonstrates feasibility and effectiveness for identifying this unique type of seal data. Moreover, compared to existing CNN model classification algorithms, this invention avoids algorithm failure when new types of seal data are introduced, achieving a one-time training and permanent use effect, thus reducing the complexity of network training to some extent. Attached Figure Description

[0016] Figure 1 Algorithm flowchart of an embodiment of the present invention.

[0017] Figure 2 Seal image sample in this embodiment of the invention.

[0018] Figure 3 Schematic diagram of preliminary rotation correction based on gradient histogram.

[0019] Figure 4 A twin network structure based on the spatial transformation layer STN.

[0020] Figure 5 A schematic diagram of a network for spatial transformation of the spatial transformation layer.

[0021] Figure 6 The accuracy of this invention in identifying seals is compared with that of other methods. Detailed Implementation

[0022] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The specific steps of the seal recognition method based on twin networks and spatial transformation are described as follows: Figure 1 As shown:

[0023] Step 1: Select a standard seal image from the reserved seal template set, denoted as O, and map the standard seal image O into a gradient histogram to obtain the horizontal coordinate position X_o of the main peak of the histogram.

[0024] Step 2: Filter the image to be corrected (median filtering) to obtain the image D to be corrected. Map the image D to be corrected into a gradient histogram to obtain the horizontal coordinate position X_d of the main peak of the histogram.

[0025] Step 3: Obtain the rotation angle A = offset * b by using the offset of the main peak of the histogram offset = X_o - Xd, where b is the angle difference, and finally obtain the corrected image of the seal to be identified, Dj.

[0026] Step 4: Build a basic twin network N, which consists of a series of convolutional layers with two identical structures.

[0027] Step 5: Add a spatial transformation layer at the beginning of the Siamese network N input and modify the loss function, changing the adversarial loss function to binary cross entropy, thus obtaining the spatial transformation network. The input of the spatial transformation network is 105*105 and the output is 105*105.

[0028] Step 6: Train the spatial transformation network.

[0029] Step 7: Input the corrected seal image D_j obtained in Step 3 into the trained spatial transformation network to finally obtain the seal recognition result.

[0030] Step 1: Select a standard seal image from the reserved seal template set, denoted as O. Map the standard seal image O into a gradient histogram to obtain the horizontal coordinate position X of the main peak of the histogram. o First, the original image is convolved using the [-1,0,1] gradient operator to obtain the gradient component g in the x-direction (horizontal direction, with the rightward direction as positive). x Then, the original image is convolved using the [1,0,-1]T gradient operator to obtain the gradient component g_y in the y-direction (vertical direction, with upward as the positive direction). The gradient magnitude G(x,y) and direction a(x,y) of the pixel are calculated using formula [1].

[0031]

[0032] Histogram plotting in HOG. The horizontal axis of the histogram consists of 60 bins. Since the image rotation range is 0 to 360 degrees, each bin represents the range of angles corresponding to the image gradient direction from x to x+6, where X = 6N, N ∈ (0, 59). The vertical axis of the histogram is the sum of the image gradient magnitudes within that bin. Figure 3 As shown.

[0033] Step 2: Filter the signature image to be corrected to obtain signature image D. Map signature image D to a gradient histogram to obtain the horizontal coordinate position X_d of the main peak of the histogram. The specific method is the same as in Step 1. Figure 3 As shown.

[0034] In this embodiment, the imprint image to be corrected is filtered using a median filtering method.

[0035] Step 3: Obtain the rotation angle A = offset * b, where b is the angle difference, by using the offset of the main peak's horizontal coordinate offset = X_o - X_d from the histogram. In this embodiment, b = 6. Finally, the corrected image of the seal to be identified, D_j, is obtained. (Example: ...) Figure 3 As shown.

[0036] Step 4: Construct a basic Siamese network N, consisting of a series of convolutional layers with two identical architectures. The Siamese network proposed in this method consists of a series of convolutional layers with two identical architectures. Each layer uses a single channel with a variable-sized convolutional kernel with a stride of 1, and the number of kernels is a multiple of 16. The network maps output characteristics using the ReLU activation function, followed by max pooling with a 2x2 pooling layer and a stride of 2. The units of the last convolutional layer are mapped to a 4096-dimensional vector by a fully connected layer. Since the network structure is two-way, two 4096-dimensional vectors are obtained. The L1 distance is calculated and finally input into the last Sigmoid activation function. Figure 4 As shown.

[0037] Step 5: Add a spatial transform net at the beginning of the Siamese network N input. The spatial transform net has an input of 105*105 and an output of 105*105. The spatial transform net is a differentiable module that performs a specific 2D affine transformation on an input feature map during the network's forward pass to obtain the transformed output feature map. For example... Figure 5 As shown, it mainly consists of three parts: 1. Localization net; 2. Gridgenerator; 3. Sampler. Figure 4As shown. The localization network obtains a feature map U of size W*H*C, and outputs θ, where θ is the parameter of the transformation used on the input feature map. For a 2D affine transformation, θ is a matrix M with 6 parameters. θ = floc(θ), and the floc function of the localization network is a fully connected layer that maps to an affine matrix. The output of the mesh generator is a parameterized sampling mesh, which is the mesh of the output feature map V. First, a mesh based on the input feature map is generated, and then the transformation T obtained in the previous step is applied. θ (G) Obtain the grid corresponding to the output feature map. The sampler can obtain values ​​corresponding to the output feature map on the input feature map. A differentiable sampling algorithm is used, generally a bilinear interpolation algorithm. The requirement that both sampling and interpolation are differentiable is to ensure that during backpropagation, gradient updates not only transmit back the input feature map but also the sampled grid coordinates. From these coordinates, partial derivatives with respect to the affine transformation parameters can be obtained, thereby optimizing the affine transformation matrix and obtaining an affine transformation suitable for seal classification. This ultimately completes the identification of seal types and improves the recognition accuracy.

[0038] The spatial transformation process can be described by formula [x]:

[0039]

[0040] Where the matrix It is a rotation matrix defined for rotation and scaling transformations.

[0041]

[0042] It is a translation vector that simply defines the offset along the horizontal and vertical axes.

[0043] Step 6: Modify the loss function, changing the adversarial loss function to binary cross entropy. The calculation formula is as follows:

[0044]

[0045] Where y i ={0,1} is a binary label, p(y i ) is the probability of the output belonging to the label y. For example... Figure 5 As shown.

[0046] Step 7: Input the corrected seal image D_j obtained in Step 3 into the trained network N to obtain the seal recognition result. Specifically, assume there is a seal data D_j to be tested, and we want to classify it into one of C categories. We already have images of C different seal templates. Each column vector represents a category. Let D_j and O_j represent a category. c Let c = 1, ..., C, and let C be the input to the similarity algorithm. Then, the algorithm predicts the category based on the maximum similarity. * =argmax c p (c) . Figure 6 The accuracy of this invention compared with other methods for identifying seals is shown in the comparison results. The results demonstrate that the technical solution used in this embodiment, namely the fifth group of Siamese NN+STN, significantly improves accuracy compared to the other technical solutions.

Claims

1. A seal recognition method based on Siamese networks and spatial transformation, characterized in that, Includes the following steps: Step 1: Obtain the seal template set, and select a standard seal image from the seal template set, denoted as O. Map the standard seal image O into a gradient histogram to obtain the horizontal coordinate position X_o of the main peak of the gradient histogram. Step 2: Filter the image to be corrected to obtain the image D to be corrected. Map the image D to be corrected into a gradient histogram to obtain the position X_d of the main peak of the gradient histogram. Step 3: Obtain the rotation angle A = offset * b by using the offset of the main peak horizontal coordinate of the gradient histogram offset = X_o - X_d, where b is the angle difference, and finally obtain the corrected image of the seal to be identified, D_j. Step 4: Build a basic Siamese network N, consisting of a series of convolutional layers with two identical structures; Step 5: Add a spatial transformation layer at the beginning of the Siamese network N input and modify the loss function, changing the adversarial loss function to binary cross-entropy, thereby obtaining the spatial transformation network. The input of the spatial transformation network is 105*105 and the output is 105*105. Step 6: Train the spatial transformation network; Step 7: Input the corrected image D_j of the seal to be identified obtained in Step 3 into the trained spatial transformation network to finally obtain the seal recognition result.

2. The seal recognition method based on twin networks and spatial transformation according to claim 1, characterized in that, The specific process of step 1 is as follows: First, perform a convolution operation on the original image using the [-1,0,1] gradient operator to obtain the gradient component g in the x-direction. x Then, the original image is convolved using the [1,0,-1]T gradient operator to obtain the gradient component g_y in the y-direction. The gradient magnitude G(x,y) and direction a(x,y) of each pixel in the image are calculated using the following formula. G y (x,y)=H(x,y+1)-H(x,y-1) G x (x,y)=H(x+1,y)-H(x-1,y) In HOG, a gradient histogram is plotted. The horizontal axis of the gradient histogram consists of 60 bins. Since the image rotation range is 0 to 360 degrees, each bin represents the range of angles corresponding to the gradient direction of the image from x to x+6, where X = 6K, K ∈ (0, 59). The vertical axis of the gradient histogram is the sum of the image gradient magnitudes within this bin. The position X_o of the main peak of the gradient histogram is obtained from the plotted gradient histogram.

3. The seal recognition method based on twin networks and spatial transformation according to claim 2, characterized in that, In the Siamese network N constructed in step 4, each layer uses a single channel with a variable-sized convolutional kernel with a stride of 1. The number of convolutional kernels is a multiple of 16. The Siamese network N maps the output characteristics using the ReLU activation function, followed by max pooling. The pooling layer has a size of 2*2 and a stride of 2. The units of the last convolutional layer are mapped to a 4096-dimensional vector by a fully connected layer. Since the network structure is two-way, two 4096-dimensional vectors are obtained. The L1 distance is calculated and finally input into the last Sigmoid activation function.

4. The seal recognition method based on twin networks and spatial transformation according to claim 1, characterized in that, The spatial transformation layer includes a positioning network, a mesh generator, and a sampler.

5. The seal recognition method based on twin networks and spatial transformation according to claim 4, characterized in that, The spatial transformation layer implements spatial transformation as follows: a feature map U of size W*H*C is obtained through a localization network, and the output is θ, where θ is the parameter used for the transformation on the input feature map. For 2D affine transformation, θ is a matrix M with 6 parameters, θ=f loc (θ), f of the localization network loc The function is a fully connected layer, mapped to an affine matrix; The output of the grid generator is a parameterized sampling grid, which is the grid of the output feature map V. First, a grid based on the input feature map is generated, and then the grid corresponding to the feature map is obtained through the affine matrix obtained by the fully connected layer of the localization network. The sampler can obtain the corresponding values ​​of the output feature map on the input feature map. The sampler adopts a differentiable sampling algorithm, which requires that both sampling and interpolation are differentiable. The gradient update can not only return the input feature map, but also the sampling grid coordinates. From the grid coordinates, the partial derivatives of the affine transformation parameters can be obtained, thereby optimizing the affine transformation matrix.

6. The seal recognition method based on Siamese networks and spatial transformation according to claim 5, characterized in that, The spatial transformation layer implements the spatial transformation process as follows: Where the matrix It is a rotation matrix defined by rotation and scaling transformations, T θ (G) The affine matrix obtained through the fully connected layer of the localization network. It is a translation vector that simply defines the offset along the horizontal and vertical axes.

7. The seal recognition method based on Siamese networks and spatial transformation according to claim 3, characterized in that, The method for modifying the loss function in step 5 is as follows: the adversarial loss function is changed to binary cross-entropy, and its calculation formula is: Where y i ={0,1} is a binary label, p(y i ) is the probability of the output belonging to the label y.

8. The seal recognition method based on Siamese networks and spatial transformation according to claim 7, characterized in that, The specific process of step 7 is as follows: Suppose we have a seal image D_j to test, and we want to classify it into one of C categories. We already have C different seal template images. Each column vector represents a category, with D_j and O_j as the two categories. c c = 1, 2, ..., C, are used as input to a similarity algorithm, which then predicts the category based on the maximum similarity. * =arg max c P (c) .

Images