Handwritten signature adversarial sample generation method based on Involution-GAN
Adversarial samples generated by the Involution-GAN model and specific loss functions solve the security robustness of the handwritten signature authentication system, realize high-quality adversarial sample detection, and can automatically detect the reliability of existing systems.
Patent Information
- Application Number
- CN202111677008.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-31
AI Technical Summary
Existing handwritten signature authentication systems have robust security challenges and are vulnerable to adversarial sample attacks, which may be misled to bypass the authentication system.
Using the handwritten signature adversarial sample generation method based on Involution-GAN, the Involution operator and specific loss functions (such as perturbation area limiting loss function, GAN loss function and adversarial loss function) are introduced to generate and automatically detect. A confrontational sample of the reliability of existing handwritten signature identification systems.
The generated adversarial samples are of high quality, and the noise is distributed on the writing handwriting rather than blank areas, reducing the possibility of being noticed by the human eye and effectively detecting the reliability of the existing handwritten signature identification system.
Smart Images

Figure CN114399842B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of information security technology, and in particular relates to a handwritten signature adversarial sample generation method based on Involution-GAN. Background Art
[0002] Handwritten signature identification is one of the most widely used biometric authentication technologies. Due to its advantages of high efficiency, reliability and low cost, this technology plays an important role in the fields of administration, finance and criminal justice identification. Handwritten signature identification technology refers to the technology of judging the authenticity of handwritten signatures by comparing the handwriting, stroke order, writing speed and pen pressure of the user's written signature. According to the different types of input signature data, handwritten signature identification technology can be divided into two categories: offline handwritten signature identification and online handwritten signature identification. For offline handwritten signature identification technology, the input data is mainly the image of the complete signature. The recognition system compares the static features such as handwriting and glyph of the input signature image with the user's signature stored in the database to finally determine the authenticity of the signature. This method does not rely on special input devices and is widely used in practice; online signature identification technology requires the use of professional hardware equipment such as digital handwriting tablets to collect dynamic information such as stroke trajectory, writing speed and force during the tester's writing process, and compares it with the existing signer's writing feature library to finally determine whether the test signature is a real signature.
[0003] In recent years, driven by various machine learning intelligent algorithms, the effectiveness and efficiency of handwritten signature identification technology have made unprecedented progress, which can effectively make up for the efficiency and cost problems of traditional manual auditing methods, and provide intelligent handwritten signature identification solutions for public security criminal investigation and banking and finance departments. However, machine learning is not a perfect solution in the field of handwritten signature identification, and these signature identification algorithms still face challenges in terms of security and reliability.
[0004] At present, handwritten signature identification mainly uses SigNet and its deformed neural network models, such as Figure 1 As shown in the figure. The model receives two signature images as output, and after being processed by two parallel convolutional networks with shared parameters, it outputs a scalar value to measure the direct similarity of the input signatures. Experiments on multiple public datasets have shown that SigNet has better robustness than other classification-based identification models. This design structure has also become the mainstream structure of the current intelligent handwritten signature identification system.
[0005] Existing research mainly focuses on how to improve the accuracy of the model, while ignoring the security issues of the handwritten signature identification model. Szegedy et al. found that the existing deep learning model has robustness problems and may be attacked by adversarial samples, that is, adding small perturbations to the image can mislead the neural network to make incorrect classifications. This makes it possible to use adversarial sample technology to construct special handwritten signature images to bypass the existing handwritten signature identification system based on machine learning. Summary of the invention
[0006] In view of the security issues existing in the existing handwritten signature identification system, the present invention proposes a handwritten signature adversarial sample generation method based on Involution-GAN, which can automatically generate adversarial samples of handwritten signature samples. Through the adversarial samples, the reliability of the existing handwritten signature identification system can be automatically detected.
[0007] In order to achieve the above object, the present invention adopts the following technical solutions:
[0008] A handwritten signature adversarial sample generation method based on Involution-GAN, comprising:
[0009] Constructing an Involution-GAN model; the Involution-GAN model consists of a generator, a discriminator, and a target neural network; the Involution operator is introduced into the generator;
[0010] The forged handwritten signature image is input into the constructed Involution-GAN model, and a handwritten signature adversarial sample is generated based on the constructed Involution-GAN model.
[0011] Further, including:
[0012] The generator receives the forged handwritten signature image, extracts the spatial features of the forged handwritten signature image and generates corresponding adversarial perturbation noise;
[0013] Limit the loss function L by the perturbation region region Constraining the adversarial disturbance noise;
[0014] Limit the loss function L to the perturbation area region The constrained adversarial perturbation noise is superimposed on the forged handwritten signature image to generate a handwritten signature adversarial sample, and the generated handwritten signature adversarial sample is input into the discriminator and the target neural network;
[0015] The discriminator distinguishes the generated handwritten signature adversarial sample from the input forged handwritten signature image;
[0016] Through the GAN loss function L GANTrain the discriminator and generator;
[0017] By adversarial loss function L adv Maximize the distance between the target neural network predicted label and the true label;
[0018] By L region , L GAN , L adv The constraints are used to generate the final handwritten signature adversarial sample.
[0019] Furthermore, the target neural network is a SigNet model.
[0020] Furthermore, the disturbance region limits the loss function L region for:
[0021] L region =||P*(1-bin(X))||2 Formula (2)
[0022] Where P represents the generated perturbation noise matrix; X represents the input original handwritten signature image sample; bin() is the image binarization conversion function.
[0023] Furthermore, the GAN loss function L GAN for:
[0024] L GAN =E x logD(X)+E x log(1-D(X+G(X))) Formula (1)
[0025] Where E x represents the computational expectation; X represents the input original handwritten signature image sample; D() is the discriminator function; G() is the generator function.
[0026] Furthermore, the adversarial loss function L adv for:
[0027]
[0028] Where E x represents the computational expectation; Indicates the average classification error of the target neural network; In the figure, f represents the target neural network, l represents the calculated classification error, X represents the input original handwritten signature image sample, G() is the generator function, and t represents the label of X.
[0029] Furthermore, after inputting the forged handwritten signature image into the constructed Involution-GAN model and generating a handwritten signature adversarial sample based on the constructed Involution-GAN model, the method further includes:
[0030] The reliability of the SigNet model is detected based on the generated handwritten signature adversarial examples.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] The present invention constructs an Involution-GAN model; at the same time, the loss function is optimized and the Involution operator is introduced, so that the generated adversarial perturbation noise is distributed as much as possible on the handwriting rather than the blank area, reducing the possibility of being perceived by the human eye. The handwritten signature adversarial sample obtained by the present invention has a high quality, and the reliability of the existing handwritten signature identification system can be automatically tested through the adversarial sample. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is the overall framework diagram of the SigNet model;
[0034] Figure 2 The overall framework diagram of the Involution-GAN model constructed for the embodiment of the present invention. DETAILED DESCRIPTION
[0035] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:
[0036] A handwritten signature adversarial sample generation method based on Involution-GAN, comprising:
[0037] Construct the Involution-GAN model, such as Figure 2 As shown in the figure, the Involution-GAN model consists of a generator G, a discriminator D and a target neural network T (i.e., a SigNet model); the Involution operator is introduced into the generator;
[0038] The forged handwritten signature image is input into the constructed Involution-GAN model, and a handwritten signature adversarial sample is generated based on the constructed Involution-GAN model.
[0039] Furthermore, it specifically includes:
[0040] The generator receives the forged handwritten signature image, extracts the spatial features of the forged handwritten signature image and generates corresponding adversarial perturbation noise;
[0041] Limit the loss function L by the perturbation regionregion Constraining the adversarial noise;
[0042] Limit the loss function L to the perturbation area region The constrained adversarial perturbation noise is superimposed on the forged handwritten signature image to generate a handwritten signature adversarial sample, and the generated handwritten signature adversarial sample is input into the discriminator and the target neural network;
[0043] The discriminator distinguishes the generated handwritten signature adversarial sample from the input forged handwritten signature image;
[0044] Through the GAN loss function L GAN Train the discriminator and generator;
[0045] By adversarial loss function L adv Maximize the distance between the target neural network predicted label and the true label;
[0046] By L region , L GAN , L adv The constraints are used to generate the final handwritten signature adversarial sample.
[0047] Furthermore, after inputting the forged handwritten signature image into the constructed Involution-GAN model and generating a handwritten signature adversarial sample based on the constructed Involution-GAN model, the method further includes:
[0048] The reliability of the SigNet model is detected based on the generated handwritten signature adversarial examples.
[0049] Specifically, the function of the generator G is to receive the original forged signature as input, extract the spatial features of the signature from these inputs and generate the corresponding adversarial perturbation noise, superimpose the adversarial perturbation and the original forged signature, and finally generate an adversarial sample that can mislead the signature authentication network, namely the discriminator D.
[0050] At the same time, considering that handwritten signatures are often single-channel black and white features, the spatial hierarchical features of the image are richer than the features of the channel dimension, which is of greater significance for how to generate targeted noise resistance. The present invention introduces the Involution operator in the generator to better focus on the spatial features of the image. The Involution operator is lighter and more efficient than the traditional convolution operation, and is simpler in form than self-attention. Ordinary convolution kernels have two basic characteristics: spatial invariance and channel specificity; while Involution is just the opposite, with channel invariance and spatial specificity. It shares kernels in the channel dimension, and uses spatially specific kernels in the spatial dimension for more flexible modeling. The algorithm pseudo code is shown below.
[0051]
[0052]
[0053] Discriminator D directly inherits the discriminator design structure of the original GAN model, and is used to distinguish the generated handwritten signature adversarial samples from the original forged handwritten signature images. The purpose of discriminator D is to encourage the generated instances to be indistinguishable from the data in the original class, and to enable the model to be trained in a semi-supervised manner. Since the method of the present invention uses perturbation position constraints, the generated adversarial samples themselves are highly consistent with the original forged samples. Therefore, we further reduce the number of neural network layers and parameters of discriminator D to accelerate the training process, reduce training costs, and ensure that the generated adversarial samples are of high quality. In order to train the discriminator and generator, the GAN loss function L GAN As shown in formula (1).
[0054] L GAN =E x logD(X)+E x log(1-D(X+G(X))) Formula (1)
[0055] Where E x represents the computational expectation; X represents the input original handwritten signature image sample; D() is the discriminator function; G() is the generator function.
[0056] There is a big difference between the adversarial sample generation of handwritten signature images and other color image adversarial techniques. In traditional image adversarial samples, since the image is a multi-channel color picture, the noise attracts less visual attention of people, so there is no need to constrain the distribution of adversarial noise points. In handwritten signature images, which are mainly black and white binarization or grayscale images, noise points are easier to be found. In addition to introducing the Involution operator in the generator to extract spatial features to make the noise points fall as much as possible in the key area that affects the subsequent target neural network classification, the present invention also designs an additional perturbation area limit loss function L region , to force the generator to learn how to generate noise points on the writing track as much as possible, as shown in formula (2). Pixels falling in the non-writing track area will make the entire L region The value of increases.
[0057] L region =||P*(1-bin(X))||2 Formula (2)
[0058] Where P represents the generated perturbation noise matrix, which has the same shape as the input handwritten signature image X; X represents the input original handwritten signature image sample; bin() is the image binarization conversion function; 1-bin(X) means setting the non-signature part (blank background of the image) to 1. P*(1-bin(X)) can measure the extent to which the perturbation pixels fall outside the handwritten signature.
[0059] In addition to training GAN and perturbing the region limit loss function, adversarial loss is also introduced. The main goal is to maximize the distance between the target neural network predicted label and the true label to achieve the effect of misleading the target neural network, as shown in formula (3). The final global loss function L global It is the weighted sum of GAN loss, perturbation region restriction loss and adversarial loss, as shown in formula (4). α and β are hyperparameters that balance the influence of the three.
[0060]
[0061] L global =L GAN +α Lregion +β Ladv Formula (4)
[0062] Where E x represents the computational expectation; Indicates the average classification error of the target neural network (i.e. SigNet); In the figure, f represents the target neural network, l represents the calculated classification error (i.e., cross entropy loss); X represents the input original handwritten signature image sample; G() is the generator function; and t represents the label of X (i.e., the original label).
[0063] In summary, the present invention constructs the Involution-GAN model; at the same time, the loss function is optimized and the Involution operator is introduced, so that the generated adversarial perturbation noise is distributed as much as possible on the handwriting rather than the blank area, reducing the possibility of being perceived by the human eye. The handwritten signature adversarial sample obtained by the present invention has a high quality, and the reliability of the existing handwritten signature identification system can be automatically tested through the adversarial sample.
[0064] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A handwritten signature adversarial sample generation method based on Involution-GAN, characterized in that: include: Constructing an Involution-GAN model; the Involution-GAN model consists of a generator, a discriminator, and a target neural network; the Involution operator is introduced into the generator; Input the forged handwritten signature image into the constructed Involution-GAN model, and generate handwritten signature adversarial samples based on the constructed Involution-GAN model; The method specifically includes: The generator receives the forged handwritten signature image, extracts the spatial features of the forged handwritten signature image and generates corresponding adversarial perturbation noise; Limit the loss function L by the perturbation region region Constrain the anti-disturbance noise; the disturbance area limits the loss function L region for: L region =||P*(1-bin(X))||2 Formula (2) Where P represents the generated perturbation noise matrix; X represents the input original handwritten signature image sample; bin() is the image binarization conversion function; Limit the loss function L to the perturbation area region The constrained adversarial perturbation noise is superimposed on the forged handwritten signature image to generate a handwritten signature adversarial sample, and the generated handwritten signature adversarial sample is input into the discriminator and the target neural network; The discriminator distinguishes the generated handwritten signature adversarial sample from the input forged handwritten signature image; Through the GAN loss function L GAN Train the discriminator and generator; By adversarial loss function L adv Maximize the distance between the target neural network predicted label and the true label; By L region , L GAN , L adv The constraints are used to generate the final handwritten signature adversarial sample.
2. The method for generating adversarial samples of handwritten signatures based on Involution-GAN according to claim 1, characterized in that: The target neural network is a SigNet model.
3. The method for generating adversarial samples of handwritten signatures based on Involution-GAN according to claim 1, characterized in that: The GAN loss function L GAN for: L GAN =E x logD(X)+E x log(1-D(X+G(X))) Formula (1) Where E x represents the computational expectation; X represents the input original handwritten signature image sample; D() is the discriminator function; G() is the generator function.
4. The method for generating adversarial samples of handwritten signatures based on Involution-GAN according to claim 1, characterized in that: The adversarial loss function L adv for: Where E x represents the computational expectation; Indicates the average classification error of the target neural network; In the figure, f represents the target neural network, l represents the calculated classification error, X represents the input original handwritten signature image sample, G() is the generator function, and t represents the label of X.
5. The method for generating adversarial samples of handwritten signatures based on Involution-GAN according to any one of claims 1 to 4, characterized in that: After inputting the forged handwritten signature image into the constructed Involution-GAN model and generating a handwritten signature adversarial sample based on the constructed Involution-GAN model, the method further includes: The reliability of the SigNet model is detected based on the generated handwritten signature adversarial examples.
