An offline signature authenticity detection method based on multi-feature fusion

Abstract

The application discloses a kind of offline signature authenticity detection methods based on multi-feature fusion, comprising the following steps: random equal proportion division training dataset and test dataset, signature dataset image sample pre-processing, local binary pattern feature extraction, select optimal distance measure function, Gabor filter feature extraction, select the optimal combination design of frequency and angle of Gabor convolution kernel and feature vector, shape factor feature extraction, select the series that can achieve the best effect, the features obtained by optimal solution are fused, and the final accuracy is calculated on the test set.The application combines the advantages of local binary pattern feature, Gabor filter feature and shape factor feature, and has high sensitivity to light source in picture sample, is consistent with human visual receiving field model, has improved and enhanced effect on handwriting identification in daily life and signature anti-forgery detection on parties of legal documents contract compared with other methods.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine learning, image processing and signature forgery detection, and particularly relates to an offline signature authenticity detection method based on multi-feature fusion. BACKGROUND

[0002] As one of the historical biometric identification methods, signature has more advantages than other feature identification methods in terms of uniqueness and convenience. The signature identification does not require complex electronic equipment, and usually only a pen and paper are needed to collect the signature with the style of the person to be tested. Today, reliable signature identification still depends on the personal experience of experts, which is often subjective. Moreover, due to the limited number of such experts, it is difficult to achieve widespread use in space and time.

[0003] Signature authenticity detection is essentially a classification task, and traditional classifiers have some shortcomings to some extent. As a classic binary classifier, support vector machine (SVM) is difficult to implement for large-scale training samples in practice; logistic regression is difficult to handle the problem of unbalanced data; and the decision tree model is prone to produce a too complex model, which has poor generalization performance for data.

[0004] In recent years, the emerging method of deep learning instead of traditional machine learning has achieved satisfactory detection accuracy in results, but the high requirement for hardware devices and expensive cost make the application of signature forgery detection not popular in the market. SUMMARY

[0005] In order to overcome the above shortcomings, the purpose of the present application is to provide an offline signature authenticity detection method based on multi-feature fusion. The method uses shape index self-normalization and features that can reflect local shape, and fuses the signature features extracted by shape index with the features extracted by traditional local binary pattern and Gabor filter.

[0006] To achieve the above purpose, the following technical scheme is proposed:

[0007] An offline signature authenticity detection method based on multi-feature fusion comprises the following steps:

[0008] 1) Randomly and proportionally divide the training data set and the test data set;

[0009] 2) Signature data set image sample preprocessing;

[0010] 3) Local binary pattern feature extraction, and selection of optimal distance measurement function;

[0011] 4) Gabor filter feature extraction, select the optimal combination of Gabor convolution kernel frequency, angle design and feature vector;

[0012] 5) shape factor feature extraction, select the best effect of the series;

[0013] 6) the features obtained by steps 3) to 5) are fused, and the final accuracy is calculated on the test set.

[0014] Further, step 1) specifically comprises the following steps: there are M people in the data set, each person has 20 real signature samples, a total of 20M, each person's real signature is written by two people, 8 copies each, i.e. 320M false signature samples, the real signature and the real signature are combined two by two to form S1=320M pairs of data samples, the real signature and the real signature are combined two by two to form For data samples, the total data samples S=S1+S2 pairs, the total data samples are evenly divided, then the training set sample number T1=the test set sample number T2, i.e. T1=T2=S / 2.

[0015] Further, step 2) specifically comprises the following steps:

[0016] 2.1) The sample pictures in the public data set often have table boundaries, these boundary lines are often redundant noise interference, which will have a great impact on the result, so the signature area needs to be segmented separately, the Canny operator is used to get the image edge, the Hough line detection is used, the grid point coordinates are located, and the image is segmented and the grid lines are removed according to the coordinates;

[0017] 2.2) Considering that some features of the signature are sensitive to rotation, the method of finding the longest inertia principal axis is used to correct the inclination of the sample;

[0018] 2.3) Some areas of the obtained signature handwriting have light color, and the ordinary standard global binarization method cannot clearly distinguish this part of the area, so a block-based binarization method is used to distinguish the light color handwriting and retain the light color handwriting.

[0019] 2.4) The model of the pen used for hesitation signature is different, and the thickness of the handwriting will change, although the thickness of the handwriting can reflect part of the writer's writing style, it is necessary to pay attention to the thickness of the same handwriting when focusing on the shape features of the handwriting, so the skeleton extraction operation is needed for the handwriting.

[0020] Further, step 3) specifically comprises the following steps: the field of local binary pattern is set to the center pixel point circle for sampling, and the chi-square distance measurement function is used on the circular local binary pattern feature to obtain the test result.

[0021] Further, the operation mode of step 4) is: using Gabor convolution kernel modulated by Gaussian function and cosine function, adjusting the parameters of Gabor convolution kernel to reflect the handwriting texture features of different directions and scales, and normalizing the L1 weight of Gabor convolution kernel template.

[0022] Further, the specific operation mode of normalizing the L1 weight of Gabor convolution kernel template is: sending the sample image after binarization preprocessing into Gabor filter kernel with angles of 0, π / 4, π / 2, 3π / 4 and frequency of 8, performing L1 normalization operation on the filter kernel template, obtaining a gray image with gray values between (-1, 1) after the image is subjected to the filtering operation, uniformly taking 64 levels to statistically obtain the gray histogram, and normalizing the gray distribution to calculate the probability density function and compare the chi-square distance.

[0023] Further, step 5) specifically comprises the following steps:

[0024] 5.1) calculating the shape index, removing the undefined part and only focusing on the curvature of the handwriting edge;

[0025] 5.2) similarly taking k levels to calculate the histogram of the image, and calculating the chi-square distance to distinguish the authenticity of the to-be-tested signature by comparing the distance within or outside the class.

[0026] Further, step 6) specifically comprises the following steps:

[0027] 6.1) fusing the local binary pattern feature, Gabor filter feature and shape factor feature, and adopting PCA dimension reduction;

[0028] 6.2) using the method of ensemble learning to calculate the prediction values of the three individual features after regression by weighted calculation through logistic regression.

[0029] The method has the advantages that the advantages of the local binary pattern feature, the Gabor filter feature and the shape factor feature are comprehensively combined, the method has high sensitivity to light sources in picture samples, is consistent with the human visual receiving field model, has improved and enhanced effects of handwriting identification in daily life, signature forgery detection on documents with legal effects, and the like, provides strong guarantee for the judicial identification procedure, and has great significance. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a general flowchart of the present application;

[0031] Figure 2 is an image sample preprocessing flowchart of the present application;

[0032] Figure 3is a flowchart of signature image feature extraction using Gabor filter;

[0033] Figure 4 is a chi-square distance comparison histogram of feature real and fake sample pairs extracted using Gabor filter in the present application;

[0034] Figure 5 is a three-dimensional shape diagram corresponding to the shape factor used in the present application;

[0035] Figure 6 is a flowchart of signature image feature extraction using shape index;

[0036] Figure 7 is a chi-square distance comparison histogram of shape factor feature real and fake sample pairs in the present application. DETAILED DESCRIPTION

[0037] The present application will be further described below in conjunction with examples and the accompanying drawings, but the scope of protection of the present application is not limited thereto.

[0038] Referring to Figure 1 A multi-feature fusion-based offline signature real and fake detection method, comprising the following steps:

[0039] 1) Data set collection: collect 31 people's signature handwriting, collect 20 real signatures for each person, and let another two people write 8 fake signatures each, so that a total of 9920 pairs of real and fake signature pairs can be combined. The real signature pairs and 31·20·16=9920 pairs of fake signature pairs are collected in table form, and the table unit size refers to the setting of SigComp2011, which is 59mm wide and 23mm high, and is scanned at a resolution of 400dpi.

[0040] 2) Referring to Figure 2 The signature table images collected through step 1) are preprocessed and separated into individual signature image samples, and the steps are as follows:

[0041] 2.1) First, the Canny operator is used to calculate the edges of the resulting signature form image, which includes the following steps: Gaussian filtering, gradient calculation, non-maximum suppression, bilateral thresholding, and hysteresis boundary tracking. In the non-maximum suppression step, the gradient of each pixel is set to 0 if it is not the maximum gradient in the positive and negative directions along the gradient. In the bilateral thresholding step, two empirical thresholds, th1 and th2, are set such that th1 ≤ th2. These thresholds divide the image into three parts: strong edge points (I(x, y) ≥ th2), weak edge points (th1 ≤ I(x, y) ≤ th2), and background points (I(x, y) ≤ th1). In the hysteresis boundary tracking step, only the pixels that are connected to strong edge points or indirectly connected through weak edge points within an 8-connected neighborhood are considered as edge points. This can be solved using the flood fill algorithm.

[0042] 2.2) Next, Hough line detection is performed by counting the number of foreground points on a given line and determining whether it exceeds a threshold. The equation of a line passing through a point (x, y) is given by ρ = x cos θ + y sin θ. Each pair of (ρ, θ) determines a unique line, and the presence of a line can be determined by counting the number of times (ρ, θ) appears. For each foreground point, θ is taken at equal intervals within the search range, and the corresponding ρ is calculated.

[0043] 2.3) Due to the affine-like distortion during the scanning process, the intersection points obtained by Hough line detection do not correspond one-to-one with the known ideal grid point coordinates. Therefore, an optimal affine inverse transformation is found, which is equivalent to solving the least squares solution of the following overdetermined equation,

[0044] (X|Y|1)W = Ideal

[0045] where Ideal represents the ideal grid point coordinates.

[0046] 2.4) To make the binarization process able to distinguish lighter handwriting, a block-based binarization method is used as an optimization. The decision of whether an anchor point is a foreground point is based on the weighted sum of the gray values within the block and a hyperparameter C.

[0047] 2.5) The method for skew correction is to find the longest principal axis of inertia and align it with the X-axis. The principal axis of inertia can be obtained by finding the eigenvector corresponding to the largest eigenvalue of the following matrix:

[0048] x c = mean(X), y c = mean(Y)

[0049]

[0050]

[0051] where i,j are the order of the center distance, x c is the mean value of the x coordinate of the foreground points, X represents the coordinate vector of the x of the foreground points, and Y represents the coordinate vector of the y of the foreground points.

[0052] 2.6) The skeleton extraction is based on the Zhang-Suen algorithm, and the 3*3 field of the anchor point is considered,

[0053]

[0054] First, the pixel points satisfying the following conditions are deleted: ① 2≤N≤6; ② S=1; ③ P2P4P6=0; ④ P4P6P8=0, wherein N represents the number of foreground points on a circumference, S represents the number of 0-1 transitions on the circumference; and then the pixel points satisfying the following conditions are deleted: ① 2≤N≤6; ② S=1; ③ P2P4P8=0; ④ P2P6P8=0, and the process is repeated until no pixel point can be eliminated in the two steps.

[0055] 3) The original LBP (Local Binary Pattern) feature with a radius of 3 is used for feature extraction, and the chi-square distance measurement function is used to test the effect on the extracted features. The original version of LBP divides the image into several small blocks, encodes the gray values of the 8-field pixels in each small block into 8-bit binary numbers, the range of which is [0, 255], then the frequency of each mode is counted to obtain a 256-bit feature vector, and finally the feature vectors of each small block are linked to obtain the feature vector of the whole image. The encoding of one pixel point is as follows:

[0056]

[0057]

[0058] wherein represents a binary number with binary bits g1g2g3g5g8g7g6g4 in turn.

[0059] The field of the original LBP is set to be a uniform sampling mode with the center pixel point as the center and a radius of 3, in order to consider the features of a larger scale, and the original LBP feature has a certain robustness to image illumination. In the experiment process of the present application, it is found that the effect of the chi-square distance is better than that of other distances.

[0060] 4) Reference Figure 3The sample image is binarized to 0 and 1, and a Gabor convolution kernel template is prepared, and the optimal Gabor parameters are adjusted, and the purpose of using the convolution kernel is that, as the real part of the Gabor filter, it is modulated by a Gaussian function and a cosine function, and the spatial domain and the time domain are considered, and by adjusting the parameters, the texture features of different scales and directions can be reflected, and the specific response function is as follows:

[0061]

[0062]

[0063] Wherein, θ represents the direction of the convolution kernel; σ reflects the window size; λ reflects the density of the texture; ψ represents the phase parameter of the cosine function in the Gabor kernel function; and γ represents the aspect ratio, which determines the ellipticity of the Gabor function shape.

[0064] In addition, the Gabor convolution kernel template L1 weight needs to be normalized, and the gray value of the image filtered by the Gabor filter is between (-1, 1), then 64 levels are uniformly taken to count the gray histogram, and the gray distribution is normalized to count the probability density function, and finally the chi-square distance is compared, in the application, the frequency of the Gabor convolution kernel is adjusted to 8, the angle is taken as 0, The feature vectors extracted by different convolution kernels are connected as the combined features. Figure 4 The L1 distance comparison of a pair of real and fake samples after the above operation is reflected.

[0065] 5) Refer to Figure 6 Firstly, the sample picture needs to be binarized, and then the shape index is calculated according to the formula, and the calculation method of the shape index is based on the Hessian matrix, and the specific method is as follows:

[0066]

[0067] Let λ1 and λ2 be two eigenvalues of the Hessian matrix, and λ1> λ2, then the shape index is defined as The shape index value reflects the shape as shown in Figure 5 .

[0068] Reference Figure 5 It can be seen that the shape index of the flat area is not defined, so the irrelevant part needs to be removed, and only the curvature of the ink edge is concerned, and finally 36 levels are taken to calculate and count the gray histogram, as shown in Figure 7 The chi-square distance of the real and fake samples of the shape index feature is compared.

[0069] The feature fusion in the application is using an integrated learning method, which is an iterative structure, specifically referring to using three predicted probability distribution values obtained by passing single features through a sigmoid function in a logistic regression as the result of a previous round of weak learners, so as to increase the weight of the data of the previous round of training errors, and then passing the output values of the weak learners through a splicing and logistic regression to obtain the accuracy of the strong classifier. In the process, the application uses a PCA (Principal Component Analysis) dimension reduction method to find an optimal linear transformation to reduce the original data to a set of orthonormal bases, so that the maximum variance can be retained. In this way, the overfitting of the training data is prevented, and the difficulty of training is reduced.

Claims

1. An offline signature authenticity detection method based on multi-feature fusion, characterized in that... Includes the following steps: 1) Randomly and proportionally divide the training dataset and the test dataset; 2) Preprocessing of image samples in the signature dataset; 3) Local binary pattern feature extraction, and selection of the optimal distance metric function; 4) Gabor filter feature extraction, selecting the optimal combination of Gabor convolution kernel frequency and angle design, and feature vector; 5) Shape factor feature extraction, selecting the level that achieves the best results; 6) Fuse the features obtained from the optimal solutions in steps 3) to 5) and calculate the final accuracy on the test set; Step 2) specifically includes the following steps: 2.1) The signature region is segmented separately, the Canny operator is used to obtain the image edges, Hough line detection is performed, grid point coordinates are located, and the image is segmented based on the coordinates and the grid lines are removed; 2.2) The sample tilt is corrected by finding the longest principal axis of inertia; 2.3) A block-based binarization method is used to identify and retain light-colored handwriting. 2.4) Perform skeleton extraction on the handwriting; Step 5) specifically includes the following steps: 5.1) Calculate the shape index, remove the undefined part, and only focus on the curvature of the handwriting edges; 5.2) Similarly, take the histogram of the k-level image and calculate the chi-square distance. Determine the authenticity of the signature to be tested by comparing the intra-class or inter-class distances. Step 6) specifically includes the following steps: 6.1) The local binary mode features, Gabor filter features, and shape factor features are fused, and PCA is used for dimensionality reduction; 6.2) Using ensemble learning, the predicted values ​​of the three individual features after regression are weighted and calculated by logistic regression.

2. The offline signature authenticity detection method based on multi-feature fusion as described in claim 1, characterized in that... Step 1) specifically includes the following steps: The dataset contains signature samples from M people, each with 20 genuine signatures, totaling 20M signatures. Each person's genuine signature is forged by two people, each with 8 forgeries, resulting in 320M fake signatures. Genuine and fake signatures are paired to form S1 = 320M pairs of data samples. Genuine signatures are also paired to form... For the data samples, the total number of data samples S = S1 + S2. If the total number of data samples is divided equally, then the number of training set samples T1 = the number of test set samples T2, that is, T1 = T2 = S / 2.

3. The offline signature authenticity detection method based on multi-feature fusion as described in claim 1, characterized in that... Step 3) specifically includes the following steps: sampling the neighborhood of the local binary pattern on the circle of the center pixel, and using the chi-square distance metric function on the circular local binary pattern features to obtain the test result.

4. The offline signature authenticity detection method based on multi-feature fusion as described in claim 1, characterized in that... Step 4) involves using a Gabor convolution kernel tuned by Gaussian and cosine functions. By adjusting the parameters of the Gabor convolution kernel, the handwriting texture features of different directions and scales are reflected, and the L1 weights of the Gabor convolution kernel template are normalized.

5. The offline signature authenticity detection method based on multi-feature fusion as described in claim 4, characterized in that... The specific operation of normalizing the L1 weights of the Gabor convolution kernel template is as follows: The sample image after binarization preprocessing is fed into a Gabor filter kernel with angles of 0, π / 4, π / 2, 3π / 4 and a frequency of 8. The L1 normalization operation of the filter kernel template is performed. After the image passes through this filtering operation, a grayscale image with grayscale values ​​between (-1, 1) is obtained. 64 levels are uniformly taken to statistically analyze its grayscale histogram, and its grayscale distribution is normalized and statistically analyzed, that is, the probability density function is calculated, and the chi-square distance is compared.

Images