Signature anti-counterfeiting identification method, device, equipment, medium and program product

Abstract

The invention provides a signature anti-counterfeiting identification method which can be applied to the technical field of artificial intelligence. The signature anti-counterfeiting identification method comprises the following steps: detecting a signature area on a signature file, and extracting a signature image from the signature area; inputting the signature image into a preset neural network model, and outputting an identification result of the signature image; wherein the preset neural network is obtained by training based on a multi-modal training set, the multi-modal training set comprises a plurality of variant images of a plurality of signatures, and the step of generating the multi-modal training set comprises the following sub-steps: obtaining original images of a plurality of signatures of a plurality of different users; performing different-scale expansion on the signed original image and adding a noise factor to obtain a plurality of variant images with different noises and different scales; and calculating the similarity between the variant images and the original image, screening the variant images with the similarity greater than a first threshold, and adding the screened variant images into a multi-modal training set. The invention further provides a signature anti-counterfeiting recognition device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a signature anti-counterfeiting identification method, device, equipment, medium and program product. BACKGROUND

[0002] In banking business, in order to ensure transaction security and customer identity verification, handwriting identification technology plays a crucial role. However, the traditional handwriting identification method mainly relies on the naked eye observation and experience judgment of experts, which is not efficient and is easily disturbed by subjective factors, thereby limiting the accuracy of the identification result.

[0003] Currently, the existing automatic handwriting identification technology mainly has the following methods: 1. Feature-based comparison method, by extracting the features of paper signature image, such as stroke thickness, direction and curvature, etc., and comparing these features with known signature samples; 2. Pattern recognition-based comparison method, such as support vector machine (SVM) and random forest algorithm, for classification and identification of signature image; 3. Deep learning-based comparison method, using convolutional neural network (CNN) to automatically extract image features for signature authenticity identification.

[0004] Although certain scientific and technological progress has been made in the field of automatic handwriting identification, most of the current technologies are specifically designed for analyzing paper signature images, and lack adaptability for electronic handwritten signature data. Due to the differences in font size features, lifting and lowering pen features, etc. between electronic signature and traditional paper signature, it is difficult for existing technologies to directly compare and identify the two effectively. SUMMARY

[0005] In view of the above problems, the present application provides a signature anti-counterfeiting identification method, device, equipment, medium and program product for improving the accuracy of signature anti-counterfeiting identification.

[0006] According to a first aspect of the present application, a signature anti-counterfeiting identification method is provided, the method comprising: detecting a signature area on a signature file, extracting a signature image from the signature area; inputting the signature image into a preset neural network model, and outputting an identification result of the signature image; wherein the preset neural network is trained based on a multi-modal training set, the multi-modal training set includes a plurality of variant images of a plurality of signatures, and generating the multi-modal training set includes: obtaining original images of a plurality of signatures of a plurality of different users; expanding the original images of the signatures to different scales and adding noise factors to obtain a plurality of variant images with different noise addition and different scales; calculating the similarity between the variant images and the original images, and selecting the variant images with similarity greater than a first threshold to join the multi-modal training set.

[0007] According to an embodiment of the present application, the generating noise factors, the expanding the signed original image to different scales and adding noise factors to obtain a plurality of variant images with different noise addition and different scales comprises: generating the original image based on a progressive growing generative adversarial network to generate a plurality of initial variant images with different resolutions from low to high; randomly generating a plurality of noise factors, generating a plurality of mixed factors by noise interpolation on the plurality of noise factors, the noise factors including gray noise, contrast noise, hue adjustment noise, angle adjustment noise and random noise; fusing the noise factors and the mixed factors with the initial variant images respectively according to a random ratio to obtain a plurality of the variant images.

[0008] According to an embodiment of the present application, the calculating the similarity between the variant image and the original image comprises: encoding the variant image and the original image based on two neural networks with the same structure and shared weights respectively to obtain a variant feature vector and an original feature vector; calculating the distance between the variant feature vector and the original feature vector to obtain the similarity.

[0009] According to an embodiment of the present application, the detecting a signature area on the signature file and extracting a signature image from the signature area comprises: extracting a bounding box coordinate on the signature file, calculating a probability that the bounding box coordinate is a signature area; when the probability is greater than a second threshold, determining the signature area and extracting the signature image.

[0010] According to an embodiment of the present application, the extracting a bounding box coordinate on the signature file and calculating a probability that the bounding box coordinate is a signature area comprises: converting a scanned image of the signature file into a gray image; scanning the gray image, finding a connected area on the scanned image by comparing pixel gray values to obtain a candidate bounding box; extracting a morphological feature of the candidate bounding box, and calculating a probability that the morphological feature matches a preset bounding box feature.

[0011] According to an embodiment of the present application, the preset neural network model comprises a deformable convolution layer in a reverse residual block, and the inputting the signature image into the preset neural network model to output an identification result of the signature image comprises: scanning the signature image based on the deformable convolution layer to obtain a plurality of stroke features; compressing the stroke features after convolution calculation based on the reverse residual block to obtain key stroke features; calculating a matching degree between the key stroke features and a pre-stored original signature, and determining the identification result of the signature image based on the matching degree.

[0012] The second aspect of the present application provides a signature anti-counterfeiting identification device, the device comprising: an image extraction module for detecting a signature area on a signature file and extracting a signature image from the signature area; an image recognition module for inputting the signature image into a preset neural network model and outputting an identification result of the signature image, the preset neural network being trained based on a multi-modal training set, the multi-modal training set comprising a plurality of variant images of a plurality of signatures; and a data set generation module for generating the multi-modal training set, comprising: an image acquisition sub-module for acquiring original images of a plurality of signatures of a plurality of different users; a noise synthesis sub-module for expanding the original images of the signatures by different scales and adding noise factors to obtain a plurality of variant images with different noise addition and different scales; and an image screening sub-module for calculating the similarity between the variant images and the original images and screening the variant images with a similarity greater than a first threshold to be added to the multi-modal training set.

[0013] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0015] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the above method. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application, taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 An application scenario diagram of a signature anti-counterfeiting identification method, device, equipment, medium and program product according to an embodiment of the present application is schematically shown;

[0018] Figure 2 A flowchart of a signature anti-counterfeiting identification method according to an embodiment of the present application is schematically shown;

[0019] Figure 3 A diagram for expanding a multi-modal training set according to an embodiment of the present application is schematically shown;

[0020] Figure 4 A structural block diagram of a signature anti-counterfeiting identification device according to an embodiment of the present application is schematically shown; and

[0021] Figure 5 A block diagram of an electronic device suitable for implementing the signature anti-counterfeiting identification method according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely exemplary and is not intended to limit the scope of the application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, mean the term "comprises," as long as the above-mentioned features, steps, operations, and / or components are present.

[0024] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0025] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include any of one, or two or more of the listed items, unless otherwise specifically defined (e.g., "a system having at least one of A, B, and C" should be interpreted to include a system having A alone, a system having B alone, a system having C alone, a system having both A and B together, a system having both A and C together, a system having both B and C together, and / or a system having all of A, B, and C together, etc.).

[0026] It should be noted that the signature anti-counterfeiting identification method and device provided by the present application can be used in the field of financial technology for signature anti-counterfeiting, and can also be used in any field other than the field of financial technology. The application field of the signature anti-counterfeiting identification method and device provided by the present application is not limited.

[0027] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user device information such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal.

[0028] The embodiment of the present application provides a signature anti-counterfeiting identification method, which comprises the following steps: detecting a signature area on a signature file, and extracting a signature image from the signature area; inputting the signature image into a preset neural network model, and outputting an identification result of the signature image; wherein, the preset neural network is trained based on a multi-modal training set, and the multi-modal training set comprises a plurality of variant images of a plurality of signatures; the generation of the multi-modal training set comprises the following steps: obtaining original images of the plurality of signatures of a plurality of different users; expanding the original images of the signatures in different scales and adding noise factors to obtain a plurality of variant images with different noise addition and different scales; calculating the similarity between the variant images and the original images, and screening the variant images with a similarity greater than a first threshold to add the variant images to the multi-modal training set.

[0029] Figure 1 An application scenario diagram of the signature anti-counterfeiting identification method, device, equipment, medium and program product according to the embodiment of the present application is schematically shown.

[0030] As shown in Figure 1 The application scenario 100 according to the embodiment can include a signature confirmation link in a financial transaction scenario. When the user needs to sign and confirm in the interaction link, for example, signs a contract, whether face-to-face signing or online signing, the authenticity of the signature can be verified first to prevent the occurrence of cases such as proxy signing or forged signature. The staff can first scan the electronic file by placing the signature file in the scanner, and then input the electronic file into the computer that can execute the signature anti-counterfeiting identification method of the present application to identify the authenticity. The neural network model used in the signature anti-counterfeiting identification method can be trained based on historical signature data in the database, and the data in the database can be continuously updated to improve the accuracy of the neural network model.

[0031] It should be noted that the signature anti-counterfeiting identification method provided in this application embodiment can generally be executed by a computer. Correspondingly, the signature anti-counterfeiting identification device provided in this application embodiment can generally be installed in a computer. The signature anti-counterfeiting identification method provided in this application embodiment can also be executed by a server or server cluster that is different from a computer and capable of communicating with scanning devices and / or computers. Correspondingly, the signature anti-counterfeiting identification device provided in this application embodiment can also be installed in a server or server cluster that is different from a computer and capable of communicating with scanning devices and / or computers.

[0032] It should be understood that Figure 1 The number of scanning devices, networks, and computers shown is merely illustrative. Any number of scanning devices, networks, and computers can be included depending on implementation needs.

[0033] The following will be based on Figure 1 The described scene, through Figures 2-3 The signature anti-counterfeiting identification method according to the embodiments of this application will be described in detail.

[0034] Figure 2 A flowchart illustrating a signature anti-counterfeiting identification method according to an embodiment of this application is shown.

[0035] like Figure 2 As shown, the signature anti-counterfeiting identification method in this embodiment includes operations S210 to S220.

[0036] In operation S210, the signature region on the signature file is detected, and the signature image is extracted from the signature region.

[0037] In this embodiment, if the signature document is paper-based, it can be scanned first using an image scanner to obtain its electronic text, facilitating the extraction of the signature image later. For the scanned electronic text, the signature can be identified by detecting specific areas of the signature document (e.g., the signature frame), thereby extracting the signature image. This method can minimize the range of the signature image, improving the accuracy of signature recognition. If the signature document is signed using an electronic signature pad, the signature pad collects the user's handwriting data in real time to generate the signature image.

[0038] In this embodiment of the disclosure, S210 includes S211 to S212.

[0039] In operation S211, the border coordinates on the signature file are extracted, and the probability that the border coordinates are the signature area is calculated.

[0040] S211 includes S2111 to S2112.

[0041] In operation S2111, the scanned image of the signature file is converted into a grayscale image. The color / monochrome scanned image is converted into a grayscale image to reduce the computational complexity while preserving the contrast between the signature and the background. If the signature and the background have a significant color difference, channel separation (such as only keeping the red channel to enhance the red signature) can be attempted. For low-contrast images, histogram equalization is performed before grayscale conversion.

[0042] In operation S2112, the grayscale image is scanned to find the connected regions on the scanned image by comparing the pixel grayscale values, and a candidate border is obtained. First, an adaptive threshold (such as the Otsu algorithm) can be used to convert the grayscale image into a binary image to highlight the signature area. Before binarization, Gaussian filtering is used for denoising to avoid interference from small regions. Then, the border that may contain the signature is found through pixel connectivity analysis. The classic algorithm for scanning connected regions is the Two-Pass algorithm. In this algorithm, regions that are too small or too large can be removed according to prior knowledge (such as the minimum / maximum size of the signature).

[0043] In operation S2113, the morphological features of the candidate border are extracted, and the probability of matching the preset border features is calculated. This step filters out the most likely signature border through geometric features. Common morphological features include the aspect ratio, the fill rate (area of the region / area of the border), the Hu moment (rotation and scaling invariance), and the edge complexity. Among them, the signature box is usually a horizontal rectangle; the signature stroke is complex, and the fill rate is low, for example, 0.1-0.3; the signature box is rotationally and scaling invariant; and the signature edge has many sawtooth edges, which can be calculated by Canny edge detection to obtain the edge point density. If the signature template is known, the similarity between the above several morphological features of the candidate border and the signature template can be calculated to obtain the probability of matching the preset border features.

[0044] In operation S212, when the probability is greater than a second threshold, the signature region is determined and the signature image is extracted.

[0045] In the embodiments of the present disclosure, the CTPN (Connectionist Text Proposal Network) algorithm can also be used to obtain the signature region. CTPN is a text detection algorithm that combines the deep learning technologies of convolutional neural networks (CNN) and long short-term memory networks (LSTM), and can effectively detect horizontally arranged text in a scene and divide the corresponding region according to the position of the text.

[0046] After obtaining the signature image, the signature image can be standardized to adapt to the neural network model used subsequently. Standardizing the signature image includes image binarization, denoising, cropping, size normalization, and color normalization, etc.

[0047] In operation S220, the signature image is input into a preset neural network model, and an identification result of the signature image is output.

[0048] In the embodiments of the present application, the preset neural network is trained based on a multi-modal training set, and the multi-modal training set includes multiple variant images of multiple signatures. The variant images may be, for example, stroke deformation, angle change, brightness change, local lightness change, stroke weight change, note thickness change, color depth change, ink penetration, intellectual disability texture change, and the like. Training the neural network model through multi-modal variant images can make the model adapt to different input conditions, force the model to focus on essential features (such as handwriting topological structure) through diversity, be more fault-tolerant to degradation phenomena in real scenes, and improve the ability of the model to identify non-natural handwriting features.

[0049] Generating the multi-modal training set includes S221-S223.

[0050] In operation S221, original images of multiple signatures of multiple different users are obtained.

[0051] In the embodiments of the present application, the original images of the signatures of the users can be obtained in the daily business operations of the users, for example, collected from paper documents signed by the users, electronic signature boards, mobile terminals, and the like. In order to further improve the accuracy of the signatures of the users, for each user, multiple original images of signatures can be collected. It should be noted that the collection of the signatures of the users needs to be performed after the user authorization, for example, a request for obtaining signature data of the user can be sent to the user before operation S221. In the case where the user agrees or authorizes to obtain the signature data of the user, the signature data of the user is stored in a database for execution of S221.

[0052] In operation S222, the original images of the signatures are expanded in different scales and added with noise factors to obtain multiple variant images with different noise addition and different scales.

[0053] In the embodiments of the present application, expanding the original images of the signatures in different scales can simulate the morphological changes of the signatures under different resolutions or writing tools (such as handwriting thickness, ink diffusion, etc.), simulate the imaging noise in real scenes (such as sensor noise, paper texture, ink unevenness), and generate diversified variants in combination with multi-scale and noise. Different scale images (such as 0.5x, 1.0x, 2.0x original size) can be generated using a Gaussian pyramid or a Laplacian pyramid, the signature handwriting is expanded with a structural element (such as a circular kernel) to simulate a thick pen effect, Gaussian noise is used to simulate electronic noise of a scanner or a camera, and salt and pepper noise is used to simulate paper stains or ink missing, and the like. For the expanded images in different scales, different noise addition or mixed addition can be used to obtain further expanded variant images.

[0054] In operation S223, the similarity of the variant image and the original image is calculated, and the variant image with a similarity greater than a first threshold is selected to join the multi-modal training set.

[0055] In the embodiments of the present application, according to the characteristics of signature image binarization / gray scale, structure sensitivity, the similarity is evaluated by calculating the structural similarity, mean square error, intersection over union, perceptual similarity and the like. The variant with too high similarity (such as SSIM > 0.95) may cause the model to fail to learn the anti-interference ability, and part of the medium similarity samples need to be retained. For large-scale data, the similarity can be pre-screened by low resolution, and then screened in full resolution. By calculating the similarity of the variant image and the original image, and screening the samples with a similarity higher than the threshold, it can be ensured that the images after data enhancement not only maintain the authenticity (close to the original signature), but also have diversity (including noise, scale change, etc.).

[0056] According to the signature anti-counterfeiting identification method provided by the embodiments of the present application, the multi-modal training set is expanded, which helps to improve the accuracy of the neural network model in identifying signatures; the multi-modal training set can integrate various deformations of paper-based handwritten signatures and electronic signatures, which helps to train a more generalized model, thereby enhancing the ability to prevent signature forgery.

[0057] Figure 3 An illustrative diagram of expanding the multi-modal training set according to the embodiments of the present application is shown.

[0058] As shown in Figure 3 , the specific implementation of expanding the multi-modal training set is: inputting the signature image into a progressive growing generative adversarial network (PGGAN, Progressive Growing of GANs) to generate a plurality of variant images; inputting the variant image into a Siamese network to calculate the similarity between the variant image and the original image, and selecting the variant image with a similarity greater than 90% to expand the training set. The embodiments will be described in detail below.

[0059] According to operation S222, the original image of the signature is expanded in different scales and added with a noise factor to obtain a plurality of variant images with different noise addition and different scales, including operations S310-S330.

[0060] In operation S310, the original image is learned based on the progressive growing generative adversarial network to generate a plurality of initial variant images with different resolutions from low to high.

[0061] In the embodiments of the present disclosure, the original image is expanded by using the PGGAN. The progressive architecture of the PGGAN naturally has the following image expansion characteristics: 1. resolution enhancement, gradually generating high-resolution image (such as 1024x1024) details enhancement from low-resolution (such as 16x16); 2. with the increase of network layers, automatically supplementing details such as pen strokes and ink diffusion for multi-scale generation; 3. by controlling the training stage, reasonable variants of different scales can be output at the same time.

[0062] In operation S320, a plurality of noise factors are randomly generated, a plurality of mixed factors are generated by noise interpolation on the plurality of noise factors, and the noise factors include gray noise, contrast noise, hue adjustment noise, angle adjustment noise, and random noise.

[0063] Based on different noise factors, the signature image gray fluctuation, contrast change, color change, rotation angle, shooting angle change, paper texture, ink diffusion, and local stroke distortion can be simulated. Through the physical-inspired noise modeling and intelligent mixing strategy, the professional image enhancement effect can be realized on the premise of guaranteeing the signature semantic features.

[0064] In operation S330, the noise factors and the mixed factors are respectively fused with the initial variant image at a random ratio to obtain a plurality of variant images.

[0065] Fusing the plurality of noise factors and the plurality of mixed factors with the initial variant image can generate a plurality of variant images mixed with noise. The variant images can have differences in stroke strength, thickness, local brightness, color change, etc. compared with the original image, which can support the neural network model to adapt to the changes of user's handwritten signature and make more accurate judgment on the user's signature.

[0066] After obtaining enough variant images, the similarity between the variant images and the original image is calculated to filter the variant images similar to the original image to train the neural network model, which narrows down the image deformation range to improve the model accuracy. This step includes S340-S350.

[0067] In operation S340, the variant image and the original image are respectively encoded based on two neural networks with the same structure and weight sharing to obtain a variant feature vector and an original feature vector.

[0068] In operation S350, the distance between the variant feature vector and the original feature vector is calculated to obtain the similarity.

[0069] In the embodiments of the present disclosure, the similarity between the variant image and the original image is calculated based on a Siamese network. The Siamese network is a special type of neural network mainly used to determine whether two input data are similar or identical. The Siamese network is composed of two identical sub-networks that share weights, and has wide applications in the fields of face recognition, image recognition, speech recognition, etc. It maps the variant image and the original image to a feature space and calculates the distance between the two feature vectors to determine the similarity. This method is particularly suitable for handling problems with a large number of categories and a small number of samples in each category. Among them, two neural networks (such as CNN, ResNet, ViT, etc.) with the same structure and shared parameters are used to process the variant image and the original image respectively, to ensure consistent feature extraction for the two inputs and avoid coding bias. Before calculating the cosine similarity, the feature vectors are usually L2 normalized, so that the similarity depends only on the vector angle. Common similarity calculation methods include Euclidean distance, Mahalanobis distance, and cosine similarity, etc.

[0070] In operation S360, the variant image with a similarity greater than the first threshold value is screened to join a multi-modal training set, and a preset neural network model is trained based on the multi-modal training set. The multi-modal training set integrates the handwritten signature image and the synthesized variant image, which helps to train a more generalized signature recognition model, thereby enhancing the ability to prevent signature forgery.

[0071] In the embodiments of the present disclosure, the preset neural network model can be a lightweight network such as MobileNetV3. The lightweight network based on MobileNetV3 has a simple structure and high computational efficiency. Using a lightweight network for signature recognition can realize online real-time recognition and improve the real-time performance of the anti-forgery technology.

[0072] In the embodiments of the present disclosure, the deformable convolution layer is included in the inverted residual block of the preset neural network model. The sampling grid of the traditional convolution kernel is fixed (such as a regular rectangle of 3*3), which is difficult to adapt to target geometric deformation (such as scaling, rotation, non-rigid deformation) or irregular spatial distribution (such as personalized deformation of signature handwriting). By learning the offset, the sampling position of the convolution kernel is dynamically adjusted, so that the receptive field can adapt to the target shape. The inverted residual block (such as the structure in MobileNetV2) usually contains the process of expanding channels → deep separable convolution → compressing channels. In order to be efficient, the traditional inverted residual block has a small number of parameters in the middle depth convolution, and the traditional convolution can lose detailed features. The deformable convolution can capture more complex spatial relationships by dynamically adjusting the sampling points without increasing the number of parameters. In signature recognition, the local deformation of strokes (such as connected strokes and tilting) can be more accurately modeled. The traditional inverted residual block expands the channels by a factor (such as 6*), which implicitly contains multi-scale information. The deformable convolution can learn different offset patterns in different channels, thereby fusing multi-scale features within a single layer. The common problem of lightweight models is that small targets (such as small signature strokes) are easily ignored by standard convolution. The deformable convolution has focusing ability, and the offset can make the sampling points concentrate on the key areas (such as stroke intersection points), thereby alleviating information loss.

[0073] It should be noted that the deformable convolution will increase the overhead of offset calculation (about 10%-20% FLOPs), and the performance and efficiency need to be balanced. It is recommended to use it only in key layers (such as shallow layers or high layers) or to reduce the number of offset generation channels. Offset learning may be unstable at the beginning of training, and needs to be combined with gradient clipping or a small learning rate. When training is initialized, the last layer of the offset convolution can be initialized to 0, so that the initial behavior is close to the standard convolution.

[0074] When the preset neural network model is trained, the signature image is input into the preset neural network model, and the identification result of the signature image is output. This process can include S371-S373.

[0075] In operation S371, the signature image is scanned based on the deformable convolution layer to obtain a plurality of stroke features. The deformable convolution layer can dynamically capture the local deformation of signature strokes (such as connected strokes, tilting, and pressure changes), and solve the problem of insufficient modeling of non-linear deformation by traditional convolution.

[0076] In operation S372, the stroke features are compressed after convolution calculation based on the inverted residual block to obtain key stroke features. This process compresses the feature dimension under the premise of retaining discriminative information, and eliminates redundant noise (such as paper texture and scanning artifacts).

[0077] At operation S373, a matching degree of the key stroke features and the pre-stored original signature is calculated, and an identification result of the signature image is determined based on the matching degree. The process can be to calculate a similarity of the to-be-tested signature and the pre-stored original signature, and output the identification result based on the similarity. Specifically, the key stroke feature map can be flattened into a vector, a cosine similarity or an Euclidean distance is calculated, and global matching is performed; a spatial attention map (such as a Squeeze-and-Excitation module) is used to weight the key region similarity, and local matching is performed. When the similarity is greater than a threshold, it can be identified as true.

[0078] Based on the above signature anti-counterfeiting identification method, the application further provides a signature anti-counterfeiting identification device. The following will be described in detail Figure 4 The device is described in detail.

[0079] Figure 4 The structure block diagram of the signature anti-counterfeiting identification device according to the embodiment of the application is schematically shown.

[0080] As Figure 4 shown, the signature anti-counterfeiting identification device 400 of the embodiment includes an image extraction module 410, an image recognition module 420, and a data set generation module 430.

[0081] The image extraction module 410 is configured to detect a signature region on a signature file, and extract a signature image from the signature region. In an embodiment, the image extraction module 410 can be configured to perform the operation S210 described above, and details are not repeated here.

[0082] The image recognition module 420 is configured to input the signature image into a pre-set neural network model, and output an identification result of the signature image. The pre-set neural network is trained based on a multi-modal training set, and the multi-modal training set includes a plurality of variant images of a plurality of signatures. In an embodiment, the image recognition module 420 can be configured to perform the operation S220 described above, and details are not repeated here.

[0083] The data set generation module 430 is configured to generate the multi-modal training set. In an embodiment, the data set generation module 430 can be configured to perform the operation S230 described above, and details are not repeated here.

[0084] According to the embodiment of the application, the data set generation module 430 includes an image acquisition sub-module 431, a noise synthesis sub-module 432, and an image screening sub-module 433.

[0085] The image acquisition sub-module 431 is configured to acquire original images of a plurality of signatures of a plurality of different users.

[0086] The noise synthesis sub-module 432 is configured to expand the original images of the signatures in different scales and add noise factors, to obtain a plurality of variant images with different noise addition and different scales.

[0087] The image screening sub-module 433 is configured to calculate the similarity between the variant image and the original image, and screen the variant image with a similarity greater than a first threshold value to be added to the multi-modal training set.

[0088] According to an embodiment of the present application, any of the image extraction module 410, the image recognition module 420 and the dataset generation module 430 can be combined in one module, or any of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present application, at least one of the image extraction module 410, the image recognition module 420 and the dataset generation module 430 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. hardware or firmware, or in any one of software, hardware and firmware or in any appropriate combination of several of them. Alternatively, at least one of the image extraction module 410, the image recognition module 420 and the dataset generation module 430 can be at least partially implemented as a computer program module which, when executed, can perform the corresponding function.

[0089] Figure 5 A block diagram of an electronic device suitable for implementing the signature anti-counterfeiting identification method according to an embodiment of the present application is schematically shown.

[0090] As shown in Figure 5 The electronic device 500 according to an embodiment of the present application includes a processor 501 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset, and / or a special-purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 501 can also include an on-board memory for cache use. The processor 501 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present application.

[0091] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via the bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 502 and / or the RAM 503. It should be noted that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0092] According to the embodiments of the present application, the electronic device 500 can further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 can further include one or more of the following components connected to the input / output (I / O) interface 505: an input part 506 including a keyboard, a mouse, and the like; an output part 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 508 including a hard disk, and the like; and a communication part 509 including a network interface card such as a LAN card, a modem, and the like. The communication part 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 510 as necessary, so that a computer program read therefrom is installed in the storage part 508 as necessary.

[0093] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0094] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this application, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories such as the ROM 502 and / or the RAM 503 described above and / or one or more memory external to the ROM 502 and the RAM 503.

[0095] Embodiments of the present application also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the signature anti-counterfeiting identification method provided by the embodiments of the present application.

[0096] The above functions defined in the system / device of the embodiments of the present application are performed when the computer program is executed by the processor 501. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0097] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on a network medium, and be downloaded and installed through the communication part 509, and / or installed from the detachable medium 511. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

[0098] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0099] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. Programming languages include, but are not limited to, Java, C++, python, "C", or the like. Program code can execute entirely on a user's computing device, partly on the user's device, as a stand-alone software package, partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0100] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0101] Those skilled in the art will understand that features recited in the various embodiments of the present application can be combined and / or integrated in various combinations and / or permutations, even if such combinations and / or permutations are not expressly noted in the present application. In particular, features recited in the various embodiments of the present application can be combined and / or integrated in various combinations and / or permutations without departing from the spirit and teachings of the present application. All such combinations and / or integrations are within the scope of the present application.

Claims

1. A signature forgery identification method, characterized by, The method comprises: detecting a signature area on a signature file, extracting a signature image from the signature area; inputting the signature image into a preset neural network model, and outputting an identification result of the signature image; wherein the preset neural network is trained based on a multi-modal training set, the multi-modal training set comprises a plurality of variant images of a plurality of signatures, and generating the multi-modal training set comprises: obtaining original images of a plurality of signatures of a plurality of different users; expanding the original images of the signatures in different scales and adding noise factors to obtain a plurality of variant images with different noise additions and different scales; calculating the similarity between the variant images and the original images, and screening the variant images with a similarity greater than a first threshold to add to the multi-modal training set.

2. The method of claim 1, wherein, The generation of the noise factor, the expansion of the original images of the signatures in different scales and the addition of the noise factor to obtain a plurality of variant images with different noise additions and different scales comprise: learning the original images based on a progressive growing generative adversarial network to generate a plurality of initial variant images with different resolutions from low to high; randomly generating a plurality of noise factors, performing noise interpolation on the plurality of noise factors to generate a plurality of mixed factors, the noise factors including gray noise, contrast noise, hue adjustment noise, angle adjustment noise and random noise; fusing the noise factors and the mixed factors with the initial variant images respectively according to a random ratio to obtain a plurality of the variant images.

3. The method of claim 1, wherein, The calculation of the similarity between the variant images and the original images comprises: encoding the variant images and the original images based on two neural networks with the same structure and weight sharing to obtain variant feature vectors and original feature vectors; calculating the distance between the variant feature vectors and the original feature vectors to obtain the similarity.

4. The method of claim 1, wherein, The detection of the signature area on the signature file and the extraction of the signature image from the signature area comprise: extracting the bounding box coordinates on the signature file, and calculating the probability that the bounding box coordinates are a signature area; when the probability is greater than a second threshold, determining the signature area and extracting the signature image.

5. The method of claim 4, wherein, The extraction of the bounding box coordinates on the signature file and the calculation of the probability that the bounding box coordinates are a signature area comprise: converting a scanned image of the signature file into a gray image; scanning the gray image, finding a connected area on the scanned image by comparing pixel gray values, and obtaining a candidate bounding box; extracting the morphological features of the candidate bounding box, and calculating the probability that the morphological features match a preset bounding box feature.

6. The method of claim 1, wherein, The preset neural network model comprises a deformable convolution layer in an inverted residual block, the inputting of the signature image into the preset neural network model, and the outputting of the identification result of the signature image comprise: scanning the signature image based on the deformable convolution layer to obtain a plurality of stroke features; compressing the stroke features after convolution calculation based on the inverted residual block to obtain key stroke features; calculating the matching degree of the key stroke features with a pre-stored original signature, and determining the identification result of the signature image based on the matching degree.

7. A signature forgery identification device characterized by comprising: The device comprises: An image extraction module is configured to detect a signature area on a signature file and extract a signature image from the signature area; An image recognition module is configured to input the signature image into a preset neural network model and output an identification result of the signature image, wherein the preset neural network is trained based on a multi-modal training set, and the multi-modal training set includes multiple variant images of multiple signatures. A data set generation module is configured to generate the multi-modal training set, including: An image acquisition sub-module is configured to acquire original images of multiple signatures of multiple different users; A noise synthesis sub-module is configured to expand the original images of the signatures in different scales and add noise factors to obtain multiple variant images with different noise addition and different scales; An image screening sub-module is configured to calculate the similarity between the variant images and the original images, and screen the variant images with a similarity greater than a first threshold to be added to the multi-modal training set.

8. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instruction is executed by the processor to realize the steps of the method according to any one of claims 1-6.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instruction is executed by the processor to realize the steps of the method according to any one of claims 1-6. The computer program or instruction is executed by the processor to realize the steps of the method according to any one of claims 1-6.

Images