Offline handwritten signature identification method of multi-scale feature aggregation network based on graph structure modeling
By using a multi-scale feature aggregation network based on graph structure modeling, the problem of insufficient structural feature extraction capability in existing offline handwritten signature authentication methods is solved, achieving efficient and accurate signature authenticity determination, which is applicable to fields such as finance and government affairs.
Patent Information
- Application Number
- CN202511105887.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-08
- Publication Date
- 2025-11-21
AI Technical Summary
Existing offline handwritten signature authentication methods have significant shortcomings in terms of structural feature extraction capabilities, model efficiency, and robustness, making it difficult to effectively identify complex or skillfully forged signatures.
A multi-scale feature aggregation network based on graph structure modeling is adopted. Node features are extracted through multi-scale convolutional kernels, a K-nearest neighbor graph structure is constructed, and feature aggregation is performed using MRD aggregation mechanism and residual connections to generate an overall signature representation. The authenticity of the signature is judged by cosine similarity.
It significantly improves the accuracy of signature authentication and the ability to identify forged signatures, enhances the system's adaptability and computational efficiency, and is suitable for practical application scenarios with high security and accuracy requirements.
Smart Images
Figure BDA0005538356550000061 
Figure BDA0005538356550000082 
Figure BDA0005538356550000083
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision and pattern recognition, in particular to an offline handwritten signature authentication method based on a multi-scale feature aggregation network of graph structure modeling, which is used for automatic recognition and authenticity identification of offline handwritten signature images, and belongs to the cross-technical field of image recognition and biological authentication. BACKGROUND
[0002] Offline handwritten signature authentication is one of the important research directions in the field of identity verification, and is widely used in key scenarios such as financial security, legal document authentication, and government approval. In the offline environment, the system can only determine the authenticity according to the static signature image, so the model is required to have high feature discrimination ability and good generalization performance. Since the signature has a highly personalized writing style, stroke coherence and spatial structure features, its recognition task is more challenging than general image classification.
[0003] At present, the mainstream technical path adopts a deep learning method based on a convolutional neural network (CNN) or a visual Transformer (ViT) to automatically extract local features such as texture and edge from the signature image. Although the CNN method performs well in local detail modeling, it is difficult to capture the global structural relationship between strokes, and is often affected by background redundancy. The ViT introduces a global attention mechanism and has a certain structure modeling capability, but its model parameters are large and the training cost is high, and it is not optimized for the characteristics of signature images that are "strong in structure, weak in texture, and sparse in background". In addition, existing methods generally lack explicit modeling mechanisms for handwriting trajectory structures, and the discrimination ability for complex or skilled forged signatures is still insufficient.
[0004] Therefore, the existing offline handwritten signature authentication method still has obvious shortcomings in structural feature extraction capability, model efficiency and robustness. In view of this situation, it is necessary to design an offline handwritten signature authentication method based on a multi-scale feature aggregation network of graph structure modeling, which can effectively extract feature representations with discriminability under a small parameter size, thereby improving the discrimination ability of the system for signature authenticity, and taking into account the computational efficiency and practical deployment feasibility. SUMMARY
[0005] The purpose of the present application is to provide an offline handwritten signature authentication method based on a multi-scale feature aggregation network of graph structure modeling, which aims to solve the problems of weak structure modeling capability, insufficient recognition accuracy, and poor robustness to forged signatures of existing offline signature recognition methods, thereby improving the accuracy, system adaptability, and recognition ability to forged attacks of the signature authentication task.
[0006] To achieve the above purpose, the present application provides an offline handwritten signature authentication method based on a multi-scale feature aggregation network of graph structure modeling, comprising the following steps:
[0007] S1. Preprocess the original offline signature image, including image grayscale conversion, noise reduction, and image center alignment;
[0008] S2. Construct and record signature pairs based on the preprocessed signature images described above;
[0009] S3. Perform structural modeling on the preprocessed signature image and construct a graph structure;
[0010] S4. Based on the graph structure obtained above, extract multi-scale node features and perform structural aggregation to generate an overall signature representation.
[0011] S5. Calculate the similarity based on the feature vectors obtained above and output the result of the signature authenticity judgment.
[0012] The specific method for preprocessing the original offline signature image is as follows:
[0013] S11. Read the offline signature dataset containing real signatures and forged signature images;
[0014] S12. Perform a uniform format conversion on the read image, converting it to a grayscale image to reduce channel dimension redundancy;
[0015] S13. Apply Gaussian filtering to the grayscale image to remove local noise and improve image edge quality;
[0016] S14. Use the OTSU algorithm to perform threshold segmentation on the image, dividing the image into a foreground signature region and a background region;
[0017] S15. Calculate the centroid of the image pixels and perform a translation operation accordingly to center the signature text in the image center position, thereby improving the consistency of the image layout.
[0018] The specific method for constructing and recording the signature pair is as follows:
[0019] S21. Divide the signers into training set, validation set and test set to ensure sample independence and generalization ability evaluation during the experiment.
[0020] S22. Combine all real signatures of the same user in pairs to generate real-real signature pairs, which are used to train or verify the model's ability to distinguish authenticity.
[0021] S23. Combine the real signature of the same user with the corresponding forged signature to generate a real-forged signature pair, which is used to evaluate the model's detection performance against forgery attacks.
[0022] S24. To ensure sample class balance, the generated real-forged signature pairs are randomly sampled to ensure that their number is consistent with that of real-real signature pairs, thus avoiding class bias during training.
[0023] The specific method for modeling the graph structure is as follows:
[0024] S31. Use multi-scale convolution kernels (including 3×3, 5×5, and 7×7) to extract local features and obtain node features under different receptive fields.
[0025] S32. Multi-scale features are concatenated by channels and fused by 1×1 convolution to generate feature representations for each node.
[0026] S33. Construct a K-nearest neighbor graph structure based on the Euclidean distance between nodes. Background regions are automatically removed during the graph structure construction process because their node feature values are approximately zero.
[0027] The specific method of feature aggregation is as follows:
[0028] S41. Input the graph structure into the backbone module of the graph neural network, which includes multiple graph convolutional layers and feedforward network layers.
[0029] S42. In each graph convolutional layer, the MRD aggregation mechanism is used to calculate the maximum difference between the current node features and the neighbor features, the mean of the neighbor features, and the standard deviation of the neighbor features, and then the three are concatenated with the original node features to form aggregated features.
[0030] S43. Use residual connections to avoid overly smooth features and improve information flow efficiency.
[0031] S44. Perform global average pooling on the final node features to generate a whole graph feature vector, which is then output as the signature feature.
[0032] The signature determination method is as follows:
[0033] S51. Compare the two signature images and extract their graph-level feature vectors respectively.
[0034] S52. Calculate the cosine similarity between two vectors as the similarity score.
[0035] S53. Compare the similarity score with a set threshold. If the score exceeds the threshold, it is determined to be a real pair; if it is below the threshold, it is a fake pair.
[0036] This invention effectively enhances the structural information representation capability of signature handwriting through a graph-based multi-scale feature modeling approach, significantly improving the system's accuracy in distinguishing genuine from counterfeit signatures in complex scenarios. Compared with traditional texture-based recognition methods, this invention offers higher interpretability, discriminative power, and deployment adaptability, making it particularly suitable for practical applications with high security and accuracy requirements, such as government affairs, finance, and official documents. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.
[0038] Figure 1 This is a flowchart illustrating the present invention;
[0039] Figure 2 This is a schematic diagram of the network structure proposed in this invention;
[0040] Figure 3 This is a schematic diagram of the multi-scale node feature embedding module proposed in this invention;
[0041] Figure 4 This is a schematic diagram of connecting nodes in a signature image;
[0042] Figure 5 This is a histogram of the distribution of signature pair feature distances extracted from the GPDSsynthetic1000 dataset by this invention, where blue represents real-fake sample pairs and red represents real-real sample pairs;
[0043] Figure 6 This is a performance comparison chart of the present invention with other methods on the BHSig dataset;
[0044] Figure 7 This is a performance comparison chart of the present invention with other methods on the GPDSsynthetic-1000 dataset;
[0045] Figure 8 This is a performance comparison chart of the present invention with other methods on the UTSig dataset; Detailed Implementation
[0046] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0047] Example 1
[0048] Please see Figures 1 to 7 This invention provides an offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure, comprising the following steps:
[0049] S1. Preprocess the original offline signature image;
[0050] The specific method is as follows:
[0051] S11. Read the raw image data.
[0052] Specifically, the original image data is an offline signature image sample in JPG or TIF format, which comes from public datasets (such as GPDSsynthetic1000, BHSig, UTSig, etc.) or a self-built signature acquisition system.
[0053] S12. Convert the format of the original image data to obtain a grayscale image, reduce redundant data, and improve the efficiency of subsequent processing.
[0054] S13. Perform Gaussian filtering on the grayscale image, with the parameter set to σ = 0.2, to smooth the image edges and remove background noise.
[0055] S14. Use the OTSU algorithm to perform binarization segmentation on the image, dividing the image into foreground (signature handwriting) and background regions.
[0056] Specifically, by adaptively calculating the global binarization threshold of the image using the maximum inter-class variance method, the signature is separated from the background, effectively eliminating background noise and improving feature contrast.
[0057] S15. Calculate the centroid coordinates of the handwriting region in the image, and center the signature at the center of the image using an affine transformation.
[0058] This operation belongs to the translation transformation in affine transformations, and the specific steps include:
[0059] Calculate the centroid coordinates (x, y) of the foreground region (i.e., the signature). c y c ), and calculate the center coordinates (x0, y0) of the image.
[0060] Its translation amount is:
[0061] Δx=x0-x c Δy=y0-y c
[0062] Perform an affine transformation on the image:
[0063]
[0064] This preprocessing operation can unify the spatial arrangement of images, enhance the model alignment robustness, and provide input images with clear structure and standard alignment for subsequent graph structure modeling.
[0065] S2. Construct and record signature pairs based on the preprocessed signature images described above.
[0066] The specific method is as follows:
[0067] S21. Divide the signers into training set, validation set and test set;
[0068] S22. Combine all real signatures of the same user to obtain all real-real signature pairs;
[0069] S23. Combine all real and forged signatures of the same user to obtain all real-forged signature pairs;
[0070] S24. Randomly sample all real-forged signature pairs to obtain the same number of real-forged signature pairs as real-real signature pairs, so as to ensure sample balance.
[0071] This step is used to address the class imbalance problem, ensuring that the training process does not overfit or bias against fake samples, thereby improving the model's discrimination stability and generalization performance.
[0072] S3. Perform graph structure modeling on the preprocessed signature image.
[0073] The specific method is as follows:
[0074] S31. Use multi-scale convolution kernels (3×3, 5×5, 7×7) to perform convolution processing on the image and extract node features at different scales.
[0075] S32. Concatenate the output features of multi-scale convolutions through channels and fuse the information using 1×1 convolutions to generate spatially structure-sensitive node representations.
[0076] S33. Based on the Euclidean distance between nodes, a K-nearest neighbor strategy is used to construct the graph structure. Background region nodes are automatically excluded during the graph construction process because their features are close to zero, and the constructed graph only contains valid signature areas.
[0077] This graph-based encoding method effectively preserves the stroke connections and structural form of the signature, enhancing subsequent aggregation and discrimination capabilities. The model structure diagram for this part is as follows: Figure 1 As shown, the graph structure signature is represented as follows: Figure 5 As shown.
[0078] S4. Aggregate node features based on graph structure and generate graph-level signature representation.
[0079] The specific method is as follows:
[0080] S41. Input the completed graph into the backbone module of the graph neural network. The backbone network is composed of multiple graph convolutional modules and feedforward network modules stacked together.
[0081] For a graph G, the graph convolution operation is computed as follows:
[0082] G' = F(G, W)
[0083] =Update(Aggragate(G,W) agg ),W update )
[0084] Among them, W agg and W update These are the learnable parameters for the feature aggregation and update processes, respectively. In this invention, the feature aggregation method used is the proposed MRD aggregation mechanism, and the update method is a feedforward neural network.
[0085] S42. Each graph convolution module embeds the MRD aggregation mechanism proposed in this invention, calculates the maximum neighbor difference, the mean of the neighbor features, and the standard deviation of the neighbor features of the current node, and concatenates the three with the original node features to form aggregated features.
[0086] Let the feature of the current node be x. i Its set of neighboring nodes is The corresponding neighbor feature vector is x j Feature aggregation then includes the following parts:
[0087] The formula for calculating the maximum adjacent difference is:
[0088]
[0089] The formula for calculating the mean of neighbor characteristics is:
[0090]
[0091] The formula for calculating the standard deviation of neighbor characteristics is:
[0092]
[0093] The final aggregated features obtained by splicing are:
[0094] f i =[x i ||f max (i)||f mean (i)||f std (i)]
[0095] S43. Use residual connections in each layer to alleviate the problem of overly smooth node features and improve the expressive power of graph neural networks.
[0096] Specifically, for graph features The calculation process of the updated feature X′ using the MRD feature aggregation mechanism, residual connections, and feedforward neural network (FFN) is as follows:
[0097] X′=FFN(σ(MRD(XW in ))W out +X)
[0098] Among them, W in and W out σ represents the learnable parameters of a 1×1 convolutional layer, and σ represents the activation function.
[0099] S44. Finally, the node features are subjected to global average pooling to obtain the structural feature vector of the entire image, which is used for signature matching and judgment.
[0100] S5. Perform signature similarity calculation and authenticity judgment.
[0101] The specific method is as follows:
[0102] S51. Input two signature images and extract their graph-level structure feature vectors respectively.
[0103] S52. Use cosine distance to calculate the cosine similarity distance between two feature vectors. The feature similarity between genuine and genuine signature pairs should be large, i.e., the feature distance should be small; the feature similarity between genuine and counterfeit signature pairs should be small, i.e., the feature distance should be large. Therefore, the authenticity of a signature can be distinguished based on the feature distance of the signature pair.
[0104] S53. Compare the cosine similarity distance with the preset classification threshold. If the similarity is greater than the threshold, it is determined to be a genuine signature pair; otherwise, it is determined to be a forged signature pair.
[0105] The unique feature of this invention lies in its graph structure modeling approach, which establishes graph connections based on the semantic distance between node features, avoiding the over-reliance on local textures in traditional methods. Simultaneously, it enhances the graph neural network's ability to perceive stroke distribution and spatial relationships through multi-scale node feature extraction and MRD aggregation mechanisms, achieving high discrimination accuracy with a relatively small number of parameters. Compared to traditional manual authentication methods, this invention offers advantages such as high automation, fast recognition speed, and strong ability to detect skillfully forged signatures, significantly reducing subjective interference from human intervention and improving the fairness and objectivity of the signature verification process.
[0106] Example 2
[0107] The results of applying this invention to the BHSig, GPDSsynthetic1000, and UTSig datasets are as follows: Figure 5 , Figure 6 , Figure 7 , Figure 8 As shown.
[0108] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.
Claims
1. An offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure, characterized in that, Includes the following steps: S1. Preprocess the original offline signature image, including image grayscale conversion, noise reduction, and image center alignment; S2. Construct and record signature pairs based on the preprocessed signature images described above; S3. Perform structural modeling on the preprocessed signature image and construct a graph structure; S4. Based on the graph structure obtained above, extract multi-scale node features and perform structural aggregation to generate an overall signature representation. S5. Calculate the similarity based on the feature vectors obtained above and output the result of the signature authenticity judgment.
2. The offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure as described in claim 1, characterized in that, The specific methods for preprocessing the original offline signature image include: S11. Read the offline signature dataset containing real signatures and forged signature images; S12. Perform a uniform format conversion on the read image, converting it to a grayscale image to reduce channel dimension redundancy; S13. Apply Gaussian filtering to the grayscale image to remove image noise and improve image edge quality; S14. Use the OTSU algorithm to perform threshold segmentation on the image, dividing the image into a foreground signature region and a background region. S15. Calculate the centroid of the image pixels and translate accordingly to center the signature text in the image.
3. The offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure as described in claim 1, characterized in that, The specific methods for constructing and recording the signature pairs include: S21. Divide the signers into training set, validation set and test set; S22. Combine all real signatures of the same user to obtain all real-real signature pairs; S23. Combine all real and forged signatures of the same user to obtain all real-forged signature pairs; S24. Randomly sample all real-forged signature pairs to obtain the same number of real-forged signature pairs as real-real signature pairs, so as to ensure sample balance.
4. The offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure as described in claim 1, characterized in that, The specific methods for graph structure modeling include: S31. Use the multi-scale node feature embedding module to extract local image features using different convolution kernels; S32. The multi-scale output features are concatenated by channels and fused with 1×1 convolution to obtain the feature representation of each node in the figure. S33. Construct a graph structure based on the Euclidean distance between the features of the nodes, preserve semantic adjacency relationships and automatically filter background areas.
5. The offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure as described in claim 1, characterized in that, The specific methods of structural aggregation include: S41. Input the graph structure into the backbone structure of the graph neural network, which includes multiple graph convolutional modules and feedforward network modules; S42. In each graph convolution module, the MRD aggregation module is used to obtain the node neighborhood structure information. The MRD module calculates the maximum neighbor difference between the current node features and the neighbor features, the mean of the neighbor features, and the standard deviation of the neighbor features, and then concatenates the three with the original node features to form aggregated features. S43. Prevent features from being too smooth by using residual connection methods and improve the information expression ability between layers; S44. Perform global average pooling on the output node features to generate the overall feature vector of the signature image.
6. The offline handwritten signature authentication method based on a multi-scale feature aggregation network modeled with graph structure as described in claim 1, characterized in that, The signature determination methods include: S51. Input the pair of signature images to be compared, and extract the corresponding feature vectors respectively; S52. Use cosine distance to calculate the similarity score between two vectors; S53. Compare the score with the set discrimination threshold. If the similarity is higher than the threshold, it is judged as a real pair; otherwise, it is judged as a forgery.