A method for generating offline signature images with controllable stroke width

Abstract

The present invention provides a method for generating an offline signature image with controllable stroke width, comprising the following steps: obtaining a single-pixel trajectory image and an offline signature image, and preprocessing the single-pixel trajectory image and the offline signature image; calculating the stroke width level of the preprocessed offline signature image; constructing and training a recurrent generative adversarial network based on a stroke width conditional vector; and obtaining an offline signature image with controllable stroke width based on the trained recurrent generative adversarial network based on the stroke width conditional vector. The model provided by the present invention can automatically generate a large number of offline signature images with natural deformation and realistic handwriting style, alleviating the problem of insufficient training data in the field of deep offline signature authentication, and has high application value.

Description

Technical Field

[0001] The invention belongs to the technical field of pattern recognition and artificial intelligence, and in particular relates to an offline signature image generation method with controllable stroke width. Background Art

[0002] Handwritten signature is a biometric behavioral feature used for identity authentication. It can be obtained in a user-friendly and non-invasive way, so it is widely accepted by the public. Handwritten signature authentication verifies the authenticity of the signature to be tested by comparing it with the template signature of the claimed identity. Handwritten signatures usually have large intra-class differences and may be imitated by malicious forgers, which brings great challenges to the task of handwritten signature authentication. In recent years, thanks to deep learning, the performance of handwritten signature authentication has been greatly improved, but deep signature authentication still faces the problem of lack of training data from real scenes, especially skilled forged signatures. It is not always feasible to collect large-scale signature datasets, because the public and commercial use of signature data may lead to legal issues such as privacy infringement. In addition, collecting skilled forged signatures requires the imitator to carefully observe, repeatedly practice, and effectively imitate the process of real signatures, which undoubtedly consumes expensive manpower and time costs. In this case, the study of signature synthesis becomes crucial and has received increasing attention in recent years.

[0003] Compared with offline signatures, online signatures contain rich temporal information and are more likely to introduce natural deformations during the synthesis process. Therefore, synthesizing online signatures first and then converting them into offline signatures is an effective way to alleviate the lack of offline signature data.

[0004] Most of the existing online-offline signature conversion methods are based on artificial rule-based ink rendering models, which use speed information to render stroke width and pressure information to render grayscale. These methods are highly hypothetical and cannot be applied to complex real-world scenarios. Although a small number of researchers have proposed online-offline signature conversion methods based on deep learning, these methods still rely on paired online and offline datasets for training, but this special dataset is relatively rare. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an offline signature image generation method with controllable stroke width. The method has the characteristics of realistic generation effect and high degree of automation, and has high practical value.

[0006] To achieve the above object, the present invention provides an offline signature image generation method with controllable stroke width, comprising the following steps:

[0007] Obtaining a single-pixel trajectory image and an offline signature image, and preprocessing the single-pixel trajectory image and the offline signature image;

[0008] Calculating the stroke width level of the preprocessed offline signature image;

[0009] Construct and train a recurrent generative adversarial network based on stroke width conditional vectors;

[0010] Based on the trained recurrent generative adversarial network based on the stroke width conditional vector, an offline signature image with controllable stroke width is obtained.

[0011] Optionally, a method for obtaining a single-pixel trajectory image is:

[0012] Obtaining an online signature, and sampling the online signature to synthesize an online signature sequence based on a Sigma Lognormal fitting model;

[0013] The online signature sequence is rendered to obtain a single-pixel trajectory image.

[0014] Optionally, the method for preprocessing the single-pixel trajectory image and the offline signature image is:

[0015] OSTU binarization and image moment-based size normalization operations are performed on the single-pixel trajectory image and the offline signature image.

[0016] Optionally, the method for calculating the stroke width level of the preprocessed offline signature image is:

[0017] Get the signature area;

[0018] Based on Canny edge detection, obtaining the edge track length of the preprocessed offline signature image;

[0019] Based on the signature region area and the edge track length, obtaining an average stroke width of the signature;

[0020] Based on the average stroke width, a stroke width level of the preprocessed offline signature image is obtained.

[0021] Optionally, the recurrent generative adversarial network based on the stroke width condition vector includes:

[0022] Two learnable generators and two learnable discriminators;

[0023] The generator is used to generate a signature image;

[0024] The discriminator is used to distinguish the signature image generated by the generator.

[0025] Optionally, the training method for the recurrent generative adversarial network based on the stroke width conditional vector is:

[0026] A loss function is constructed, and based on the loss function, the generator and the discriminator are trained respectively, and the trained generator and the trained discriminator are optimized respectively using the Adam optimizer to complete the training of the recurrent generative adversarial network based on the stroke width conditional vector.

[0027] Optionally, the generator loss function is:

[0028] L Gen (G, F, D X , D Y )=L G，GAN (G,D Y , X)+L F，GAN (F,D X , Y)+L idt (G, F)+L cyc (G, F)+L w，Gen (G,D Y , X)

[0029] Among them, L G，GAN (G,D Y , X) is the adversarial loss function of the generator G, L F，GAN (F,D X , Y) is the adversarial loss function of the generator F, L cyc (G, F) is the cycle consistency loss function of generator G and generator F, L idt (G, F) is the input-output consistency loss function of generator G and generator F, L w，Gen (G,D Y , X) is the label-smoothed cross entropy loss function related to the stroke width corresponding to the generator G;

[0030] The discriminator loss function is:

[0031]

[0032] in, is the discriminator D Y The adversarial loss function is is the discriminator D X The adversarial loss function, L w，Disc (D Y , Y) is the discriminator D Y The corresponding label-smoothed cross entropy loss function related to stroke width;

[0033] Among them, X is the style domain of the online signature trajectory, and Y is the style domain of the offline signature image.

[0034] Optionally, based on the trained recurrent generative adversarial network based on stroke width conditional vector, a method for obtaining an offline signature image with controllable stroke width is:

[0035] Based on the trained recurrent generative adversarial network based on stroke width conditional vectors, the open set online signatures are handwritten stylized to obtain offline signature images with controllable stroke widths.

[0036] Compared with the prior art, the present invention has the following advantages and technical effects:

[0037] The present invention utilizes the powerful learning ability of neural networks to train neural networks to learn the conversion from online signature trajectories to offline handwriting style signatures without relying on manually designed rule-based ink rendering models, and has stronger generalization.

[0038] The present invention introduces an unsupervised style transfer method based on a cyclic generative adversarial network, so that the neural network can automatically learn to convert the online signature trajectory into a realistic offline signature image with a handwritten style in the absence of a paired online and offline signature training data set, and the generated effect is also closer to the distribution of the real offline signature data.

[0039] The method of the present invention has a realistic generation effect and can also combine the advantage of online signatures that they can easily synthesize naturally deformed samples to automatically generate a large number of naturally deformed and realistic offline signature images in handwriting style, thereby alleviating the problem of insufficient training data in the field of deep offline signature authentication and effectively improving the performance of the deep offline signature authentication model.

[0040] The method of the present invention can be applied to other offline handwriting generation tasks, such as stylizing the offline handwriting of online handwritten numbers, letters, and Chinese characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 This is a schematic flow chart of a method for generating an offline signature image with controllable stroke width according to the first embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of an example of offline signature generation according to the first embodiment of the present invention; DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Embodiment 1

[0047] like Figure 1 As shown, the present invention provides an offline signature image generation method with controllable stroke width, comprising the following steps:

[0048] Step 1: Use electronic handwriting equipment to collect online signature sequence, collect offline signature image by taking a photo or scanning, and process it according to the following steps:

[0049] Step 1.1: synthesize the online signature samples and render them into single-pixel trajectory images. Based on the existing SigmaLognormal fitting model, add perturbations to the fitting parameters, sample and synthesize the online signature sequence with natural deformation, and render the online signature sequence to obtain a single-pixel trajectory image; collect the offline signature image by taking pictures or scanning.

[0050] Step 1.2, perform OSTU binarization and image moment-based size normalization on the real offline signature image and the rendered single-pixel signature trajectory image. Specifically, let the image width and height be W and H respectively, and the image centroid be W(x c ,y c ), the second-order central moment is μ 20 , μ 02 , the scaling factor is determined as follows:

[0051]

[0052] And perform the following image forward mapping:

[0053]

[0054]

[0055] Where (x, y) is the coordinate of the pixel point of the original image, (x', y') is the coordinate of the pixel point of the mapped image, and 0≤x'≤W-1, 0≤y'≤H-1. After mapping, the target image with width and height of W and H respectively is obtained.

[0056] Step 2 calculates the stroke width level from the real offline signature image preprocessed in step 1. Let S be the number of foreground pixels, that is, the area of ​​the signature area. The edge trajectory length len of the signature image is obtained by Canny edge detection, and the average stroke width of the signature is It can be estimated as follows:

[0057]

[0058] The average stroke width level w∈{1,...,N} can be obtained by Round down to get the following calculation method:

[0059]

[0060] Step 3 constructs and trains the cyclic generative adversarial network Stroke-cCycleGAN based on the stroke width conditional vector, which includes the following steps:

[0061] Step 3.1, construct a cyclic generative adversarial network Stroke-cCycleGAN based on the stroke width conditional vector. Let X and Y be the style domain of the online signature trajectory and the style domain of the offline signature image, respectively, x and y are the online signature trajectory sample and the offline signature image sample, respectively, and x∈X, y∈Y. G and F are two learnable generators, corresponding to the transformation G:X→Y and the inverse transformation F:Y→X, respectively. D Y With D G There are two learnable discriminators, which are used to distinguish the signature images generated by the generator from the real signature images, while the generator is used to generate realistic signature images to deceive the discriminator. The structure of the generator follows the structure of the native cycle generative adversarial network, including a downsampling layer, an upsampling layer, and six residual blocks in the middle. In the Stroke-cCycleGAN model, the average stroke width level w is mapped into a 16-dimensional vector z, which is then sent to the latent space of the generator and trained and updated in conjunction with the generator. The structure of the discriminator is shown in the network structure of the discriminator in Table 1. The last two branches are used to determine whether a given signature image is a generated sample and predict the average stroke width level.

[0062] Table 1

[0063]

[0064] Step 3.2, construct the loss function for training the Stroke-cCycleGAN model. Two generators G, F and two discriminators D Y , D G The training is done by adversarial learning. Generator G and discriminator D YThe adversarial loss can be calculated according to Equation 6 and Equation 7 respectively:

[0065]

[0066]

[0067] Similarly, the generator F and the discriminator D X The adversarial loss can be obtained by exchanging X and Y, replacing G with F, and D Y Replace with D X Get. Cycle consistency loss L cyc Used to facilitate consistency between forward and inverse transformations, it is calculated as follows:

[0068]

[0069] Input-output consistency loss L idt Used to prevent excessive transfer of handwriting style, the calculation is as follows:

[0070]

[0071] Let W be the target stroke width space, and the discriminator D of the Stroke-cCycleGAN model Y By minimizing the label-smoothed cross entropy loss related to stroke width, we guide the generator G towards the target set Learning, generating sets The label-smoothed cross entropy loss related to the stroke width of the generator and discriminator is calculated as follows:

[0072]

[0073]

[0074] where w i ∈[1, N], p(w|·) is the output of the w-th softmax unit, and ∈=0.1 controls the degree of label smoothing. By introducing the above control conditions related to stroke width, the loss function of the generator and discriminator of the Stroke-cCycleGAN model is calculated as follows:

[0075] L Gen (G, F, D X , D Y )=L G，GAN (G,D Y , X)+L F，GAN (F,D X , Y)+L idt (G, F)+L cyc (G, F)+L w，Gen(G,D Y , X) Formula 12

[0076]

[0077] In step 3.3, the generator and discriminator of the Stroke-cCycleGAN model are trained separately by optimizing the loss function calculated in step 3.2. The Adam optimizer is used, and each training iteration is 20 times. The initial learning rate is set to 0.0002, and the number of average stroke width levels N is set to 7.

[0078] Step 4 uses the trained Stroke-cCycleGAN model to stylize the open-set online signatures and generate realistic offline signature images with rich stroke widths. In the forward process, given the target stroke width level, the generator G generates an offline signature image with realistic handwriting style and consistent with the target stroke width level based on the skeleton shape of the open-set online signature. Since the stroke width of offline signatures in real scenes is mainly concentrated in a limited range [N down ,N up ], the target stroke width level assigned to the generator should be randomly selected from this range to ensure good generation results. When different target stroke width levels are randomly assigned, the trained Stroke-cCycleGAN model can generate realistic offline signature images with rich and varied stroke widths. Figure 2 As shown, the effect of offline signature generation is shown.

[0079] The present invention proposes a method for generating offline signature images with controllable stroke width. Based on a recurrent adversarial generative network, the present invention proposes an offline signature generation model that does not require a paired online and offline signature training data set, and provides network convolution kernel size, network training method, etc., so that the model can be applied to the effective generation task of offline signatures. The model provided by the present invention can automatically generate a large number of offline signature images with natural handwriting style and realistic appearance, alleviating the problem of insufficient training data in the field of deep offline signature authentication, and has high application value.

[0080] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating an offline signature image with controllable stroke width, characterized in that: The following steps are included: Obtaining a single-pixel trajectory image and an offline signature image, and preprocessing the single-pixel trajectory image and the offline signature image; Calculating the stroke width level of the preprocessed offline signature image; Construct and train a recurrent generative adversarial network based on stroke width conditional vectors; Based on the trained recurrent generative adversarial network based on stroke width conditional vector, an offline signature image with controllable stroke width is obtained; The method for calculating the stroke width level of the pre-processed offline signature image is: Get the signature area; Based on Canny edge detection, obtaining the edge track length of the preprocessed offline signature image; Based on the area of ​​the signature region and the length of the edge track, obtaining an average stroke width of the signature; Based on the average stroke width, obtaining a stroke width level of the preprocessed offline signature image; The recurrent generative adversarial network based on the stroke width condition vector includes: Two learnable generators and two learnable discriminators; The generator is used to generate a signature image; The discriminator is used to distinguish the signature image generated by the generator; The training method for the recurrent generative adversarial network based on the stroke width conditional vector is: Constructing a loss function, training the generator and the discriminator based on the loss function, and optimizing the trained generator and the trained discriminator using an Adam optimizer to complete the training of the recurrent generative adversarial network based on the stroke width conditional vector; The loss function includes: a generator loss function and a discriminator loss function; The generator loss function is: L Gen (G,F,D X ,D Y )=L G,GAN (G,D Y ,X)+L F,GAN (F,D X ,Y)+L idt (G,F)+L cyc (G,F)+L w,Gen (G,D Y ,X), Among them, L G,GAN (G,D Y ,X) is the adversarial loss function of the generator G, L F,GAN (F,D X ,Y) is the adversarial loss function of the generator F, L cyc (G, F) is the cycle consistency loss function of generator G and generator F, L idt (G, F) is the input-output consistency loss function of generator G and generator F, L w,Gen (G,D Y ,X) is the label-smoothed cross entropy loss function related to the stroke width corresponding to the generator G; The discriminator loss function is: in, is the discriminator D Y The adversarial loss function is is the discriminator D X The adversarial loss function, L w,Disc (D Y ,Y) is the discriminator D Y The corresponding label-smoothed cross entropy loss function related to stroke width; Among them, X is the style domain of the online signature trajectory, and Y is the style domain of the offline signature image.

2. The method for generating an offline signature image with controllable stroke width according to claim 1, characterized in that: The method to obtain a single-pixel trajectory image is: Obtaining an online signature, and sampling the online signature to synthesize an online signature sequence based on a Sigma Lognormal fitting model; The online signature sequence is rendered to obtain a single-pixel trajectory image.

3. The method for generating an offline signature image with controllable stroke width according to claim 1, characterized in that: The method for preprocessing the single-pixel trajectory image and the offline signature image is: OSTU binarization and image moment-based size normalization operations are performed on the single-pixel trajectory image and the offline signature image.

4. The method for generating an offline signature image with controllable stroke width according to claim 1, characterized in that: Based on the trained recurrent generative adversarial network based on stroke width conditional vector, a method for obtaining an offline signature image with controllable stroke width is as follows: Based on the trained recurrent generative adversarial network based on stroke width conditional vectors, the open set online signatures are handwritten stylized to obtain offline signature images with controllable stroke width.

Images