A Handwritten Signature Verification Method and Device Assisted by Personality Personalization

Abstract

The present invention discloses a handwritten signature verification method and device assisted by personality personalization, belonging to the field of human-computer interaction. For the collected user handwritten signature samples, different classifiers are used for classification by using different features. Since the information provided by image features, trajectory features and user personality features is different, the discriminant accuracy can be improved by utilizing the diversity of different features. At the same time, the user's personality can be predicted according to the handwritten signature trajectory information of the user to personalize the user signature. The support vector machine classifier is trained supervised with data with positive and negative sample labels and personality labels, and a verification accuracy rate of 96% can be achieved.

Description

Technical Field

[0001] The present invention belongs to the field of human-computer interaction, and particularly relates to a handwritten signature verification method and device assisted by personality personalization. Background Art

[0002] Accurately and quickly authenticating each individual is a very important and complex task. The core of this task is to verify the unique identity of the user. Traditional individual user authentication methods include token-based authentication and knowledge-based authentication (References: D. Impedovo and G. Pirlo, "Automatic Signature Verification: The State of the Art," Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol. 38, pp. 609-635, 2008.). Token-based authentication includes chips with authorized user identity information, portable password generation devices, etc.; while knowledge-based authentication includes strings or passwords that users need to memorize. However, these authentication methods all have natural drawbacks. In token-based authentication methods, the tokens may be lost or even stolen; in knowledge-based authentication methods, users may forget their passwords. In recent years, with the development of artificial intelligence and computer vision technologies, verifying identity through people's biometric information has become a widely researched and applied technology (References: D. Impedovo, G. Pirlo, and R. Plamondon, "Handwritten Signature Verification: New Advancements and Open Issues," in Frontiers in Handwriting Recognition (ICFHR), 2012 International Conference on, 2012, pp. 367-372.). Biometric information has the characteristics of uniqueness, easy recognition, not forgetting, and no need to carry, and has natural advantages in the task of verifying individuals. A typical biometric information is an individual's handwritten signature. Among pure behavioral biometric characteristics, handwritten signatures and the way people write their signatures are the most socially recognized and legally accepted means of personal signature authentication (References: Plamondon R, Lorette G. Automatic signature verification and writer identification: the state of the art. Pattern Recognition. 1989;22(2):107–131.).Handwritten signatures can be used to verify personal identity in various fields such as administrative, commercial, and financial applications, such as signing credit card receipts and commercial contracts (References: Lai S, Jin L, Lin L, et al. SynSig2Vec: Learning Representations from Synthetic Dynamic Signatures for Real-World Verification[J]. 2020.), and can also be used to identify and prevent crimes or any situations involving identity theft and abuse.

[0003] However, handwritten signatures can be forged, especially when the forger has seen the name and signature shape of the person being forged. Distinguishing between real and forged signatures has become a challenging task (References: Z. Zhang, K. Wang, and Y. Wang, "A Survey of On-line Signature Verification," in Biometric Recognition. vol. 7098, Z. Sun, J. Lai, X. Chen, and T. Tan, Eds., ed: Springer Berlin Heidelberg, 2011, pp. 141-149.), and verifying signatures manually is inefficient and error-prone. Therefore, an efficient signature verification method is necessary.

[0004] At present, some methods for signature verification do not comprehensively consider various characteristic information of signatures, nor do they analyze the relationship between each characteristic information and the signature user. Therefore, bottlenecks will be encountered in the verification accuracy and user personalized analysis (Reference: Z. Zhang, K. Wang, and Y. Wang, "A Survey of On-line Signature Verification," in Biometric Recognition. vol. 7098, Z. Sun, J. Lai, X. Chen, and T. Tan, Eds., ed: Springer Berlin Heidelberg, 2011, pp. 141-149.). In fact, in the handwritten signature samples of users, the signature image, the coordinate points and time information of the signature, and the pressure information of the signature are all handwritten information that can be utilized. In addition, existing research has shown that there is an obvious connection between the personality of the signer and the trajectory characteristics of the signature (Reference: Miguel-Hurtado O, Guest R, Stevenage S V, et al. The relationship between handwritten signature production and personality traits[C] / / IEEE International Joint Conference on Biometrics. IEEE, 2014: 1-8.), and there is also relevant research using the user's handwritten signature to predict the user's personality (Aulia M R, Djamal E C, Bon A T. Personality Identification Based on Handwritten Signature Using Convolutional Neural Networks[C]Proceedings of the 5th NA International Conference on Industrial Engineering and Operations Management Detroit. 2020.). Therefore, when verifying a user's handwritten signature, the personality predicted by the trajectory is also a very important characteristic and can be used to conduct personalized profiling of the user's personality. Summary of the Invention

[0005] To overcome the deficiencies of the prior art, the present invention provides a method and device for handwritten signature verification assisted by personality personalization. The present invention can use the handwritten signature coordinates and timestamp information of the user under the touch screen to reproduce the signature trajectory and predict the user's personality, and combine the handwritten signature image information and signature trajectory information of the touch screen to verify whether the user is a real and legitimate user.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for handwritten signature verification assisted by personality personalization, the steps of which include:

[0008] Construct a handwritten signature list, the handwritten signature list includes: the handwritten signature and personality type of the target user;

[0009] For the handwritten signature to be verified of the target user, obtain the signature image and signature trajectory respectively;

[0010] Perform image classification based on the signature image, and combine the handwritten signature to obtain a first prediction result;

[0011] Obtain a second prediction result by calculating the distance between the signature trajectory and the target signature trajectory in the handwritten signature;

[0012] Perform personality type classification based on the signature trajectory, and combine the personality type to obtain a third prediction result;

[0013] Perform weighted calculation on the first prediction result, the second prediction result and the third prediction result to obtain the verification result of the handwritten signature to be verified.

[0014] Further, the performing image classification based on the signature image and combining the handwritten signature to obtain a first prediction result includes:

[0015] Perform normalization processing on the signature image;

[0016] Extract the features of the normalized image to obtain a feature vector;

[0017] Input the feature vector into an image classification model to obtain the image classification result of the signature image, wherein the image classification model is constructed based on a supervised machine learning algorithm;

[0018] Judge whether the image classification result is the label of the handwritten signature to obtain a first prediction result.

[0019] Further, the performing normalization processing on the signature image includes:

[0020] For the signature image, extract the minimum bounding rectangle of the signature for position normalization;

[0021] And,

[0022] Calculate the invariant rotation angle of orthogonal regression based on the variance of the horizontal coordinates, the variance of the vertical coordinates, and the covariance between the horizontal coordinate component and the vertical coordinate component in the signature image for direction normalization;

[0023] And,

[0024] Normalize the signature in the signature image for size normalization.

[0025] Further, the extracting features of the normalized image includes:

[0026] Extract contour features from the normalized image, where the contour features include: image contour information, image topology information, and point offset information of the image;

[0027] And,

[0028] Perform extreme value detection and feature point localization on the scale space of the normalized image, and cluster the obtained key points to obtain features composed of the number of key points.

[0029] Further, the supervised machine learning algorithms include: SVM, neural network, or decision tree model.

[0030] Further, the obtaining the second prediction result by calculating the distance between the signature trajectory and the target signature trajectory in the handwritten signature includes:

[0031] Extract the trajectory features of the signature trajectory and the target trajectory features of the target signature trajectory respectively;

[0032] Based on the signature trajectory and the target signature trajectory, construct a signature trajectory sequence and a target signature trajectory sequence respectively;

[0033] According to the trajectory features and the target trajectory features, use the DTW algorithm and the idea of dynamic programming to calculate the shortest matching distance length between the signature trajectory sequence and the target signature trajectory sequence;

[0034] Compare the shortest matching distance length with a threshold to obtain the second prediction result.

[0035] Further, the calculating the shortest matching distance length between the signature trajectory sequence and the target signature trajectory sequence according to the trajectory features and the target trajectory features, using the DTW algorithm and the idea of dynamic programming, includes:

[0036] Calculate the distance matrix of each point between the signature trajectory sequence and the target signature trajectory sequence;

[0037] Based on the distance matrix, obtain the shortest matching path, where the sum of the elements on the shortest matching path is the smallest;

[0038] Obtain the distance length of the shortest matching path.

[0039] Furthermore, perform personality type classification based on the signature trajectory, and combine the personality type to obtain a third prediction result, including:

[0040] Extract the trajectory features of the signature trajectory;

[0041] Input the trajectory features into a personality prediction model to obtain a personality type prediction result. Among them, the personality prediction model is constructed through the following steps:

[0042] Quantitatively model the personality psychological characteristics of the subjects through the Big Five Personality Inventory to obtain the personality dimensions of the subjects, and the personality dimensions include: openness, extraversion, agreeableness, conscientiousness or emotional stability;

[0043] Use the trajectory features of the subjects' signature trajectories as inputs, the personality dimensions as labels, and use the ResNet-18 convolutional neural network for five-classification training to obtain the personality prediction model;

[0044] Judge whether the personality type prediction result is the label of the personality type to obtain a third prediction result.

[0045] A storage medium stores a computer program, wherein the computer program is set to execute any of the above methods when running.

[0046] An electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor is set to run the computer program to execute any of the above methods.

[0047] The advantages and beneficial effects of the present invention are as follows:

[0048] 1) The innovation of this invention lies in considering the characteristics and advantages of different types of features for handwritten identity recognition. Different types of features can reflect specific user characteristics and are used for personal identity recognition of users. For example, handwriting can reflect personal signature personality, and image contours and key points can reflect overall image features, etc. However, the algorithms used in previous handwritten signature verification work did not consider which type of features the algorithm itself is more suitable for processing.

[0049] 2) Innovatively train a five-classification model for predicting personal personality based on key handwriting features through the corresponding categories in the five personalities that an individual is most inclined to.

[0050] 3) Through personality personalization, users can be more accurately profiled. Combining the user personality judged by the Big Five Personality Inventory, this model can uncover the implicit information between personality and handwriting habits, and use personality to assist in the judgment of identity verification, thereby improving the accuracy of recognition. Description of the Drawings

[0051] Figure 1 is the construction process of the entire handwritten verification system.

[0052] Figure 2 is the processing process for the handwriting channel. Detailed Implementation Manner

[0053] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below in conjunction with the drawings and the implementation manner.

[0054] A multi-channel handwritten signature verification method assisted by personality personalization (as Figure 1 shown), the steps of which are as follows:

[0055] (1) First, a large number of user handwritten signature samples are collected on the touch screen through a collection program, including positive samples (real user signatures) and negative samples (forged signatures of others in the name of the user). The collected data includes the x coordinate, y coordinate, and timestamp information of each sampling point of the sample. For each user whose handwritten signature is collected, the results of the Big Five Personality Inventory of the user are recorded.

[0056] (2) Image feature model: The coordinate point information is reproduced as image information, and the image is subjected to scale normalization, angle normalization, and grayscale processing. Feature calculation and selection are performed on the obtained image samples to screen out key features, such as key point features and contour features. And machine learning algorithms such as SVM are used to train the positive and negative sample images to obtain a handwritten signature image discrimination model.

[0057] (3) Trajectory discrimination model: The DTW algorithm is used to calculate the distance between real signature samples and select a template that can represent the user's signature. After obtaining the template sample, the distance threshold for determining whether it is a real signature is determined by comparing the distance with the forged signature sample.

[0058] (4) Personality prediction model: The trajectory information is reproduced according to the coordinate point information. And the trajectory features are transformed into a six-classification problem. According to the obtained user personality information, feature engineering is used to extract key trajectory features that can reflect the user's personality, and a classification model for predicting the user's personality is constructed.

[0059] (5) Use the image feature discrimination model obtained in step (2), the trajectory feature discrimination model obtained in step (3), and the personality prediction and determination model obtained in step (4) as the basis for discriminating whether a signature is a real sample. Assign an initial weight of 1 / 3 to each channel, and use the method of ensemble learning to reassign the weights of different channels. Finally, converge to obtain the final handwritten signature verification model.

[0060] The specific technical details are elaborated below.

[0061] 1. Image feature model:

[0062] (a) Preprocessing

[0063] First, standardize the signature according to factors such as the position, size, and direction of the signature.

[0064] For position standardization, only extract the minimum bounding rectangle of the signature.

[0065] For direction standardization, use the following formula to calculate the invariant rotation angle of orthogonal regression:

[0066]

[0067] where s x and s y are the variances of the horizontal and vertical coordinates respectively, and cov(x,y) is the covariance between the horizontal coordinate component and the vertical coordinate component.

[0068] For size standardization, use the following two formulas to ensure that the horizontal and vertical coordinates of the signature are mapped to the same interval.

[0069]

[0070]

[0071] where x and y are the original horizontal and vertical coordinates respectively, and x n and y n are the coordinate values obtained by normalization.

[0072] (b) Feature selection:

[0073] - Contour feature extraction: Obtain the image contour information through the findContours() function in OpenCV and store it in the form of a point vector; obtain the image topology information and store it in the hierarchy vector to record the parent contour and nested contour information; obtain the point offset information of the image and store it in the Point offset.

[0074] -SIFT Scale-Invariant Feature Transform Algorithm: By detecting the extreme values in the scale space and locating the feature points, features such as different types of key points, angles, response degrees, local gradients, and point diameters are obtained. Then, the key points are clustered using a clustering algorithm to obtain a feature vector composed of the number of key points. Finally, the contour features and SIFT features are fused into a feature vector.

[0075] (c) Classification:

[0076] Based on the obtained feature vector, supervised machine learning algorithms such as SVM, neural network, and decision tree models are used to classify the feature vectors of the samples to obtain the final discrimination model.

[0077] 2. Trajectory Discrimination Model:

[0078] As shown in Table 1, the key trajectory features that can be extracted from the coordinate information are presented.

[0079] Table 1 Trajectory Feature Extraction for Handwritten Recognition

[0080]

[0081]

[0082] Based on the obtained features, the DTW algorithm and the idea of dynamic programming are used to calculate the shortest matching distance length between different sample sequences. It is carried out in two steps. The first step is to calculate the distance matrix of each point between the two sequences. The second step is to find a path from the upper left corner to the lower right corner of the matrix such that the sum of the elements on the entire path is the smallest.

[0083] After obtaining the shortest matching distance length between each group of sequences, the signature with the smallest average shortest matching distance length to all other sequences is selected as the template signature for the corresponding user. And the threshold for discriminating whether it is a real sample is obtained by calculating the shortest matching distance length between all positive sample sequences and negative sample sequences and the template sequence respectively.

[0084] 3. Personality Prediction Model:

[0085] The personality psychological characteristics of the subjects are quantitatively modeled through the Big Five Personality Inventory, which is designed with reference to the Saucier (1994) version of the Big Five Personality Inventory. Regarding the measurement of the Big Five Personality, it includes five personality dimensions, namely openness, extraversion, agreeableness, conscientiousness, and emotional stability.

[0086] After collecting the personality data, the personality to which the subject belongs can be obtained, which is used as the label for neural network training, and the trajectory features of the handwritten data are used as the input. The ResNet-18 convolutional neural network is used to perform five-class classification on the user's trajectory signature. Among them, the number of training samples in each batch (Batch size) is 32, and the number of training iteration cycles (Epoch) is set to 50. The initial learning rate is set to 0.01, and the Adam optimization algorithm is used to dynamically adjust the learning rate. The network uses the Softmax function for classification and adopts the cross-entropy loss function to evaluate the gap between the true value and the predicted value. At the same time, L2 regularization is used to prevent overfitting during the training process, and the L2 regularization parameter is 0.0002. The neural network model obtained by convergence is the final personality prediction model for the user's handwritten signature trajectory.

[0087] The application scenarios of the present invention are introduced below.

[0088] Placing the touch screen in the daily work scenario, a certain number of participating users will use the touch screen to log in to their respective user information.

[0089] Through the above steps of collecting samples, building models, and algorithm integration, the final handwritten signature verification model assisted by personality personalization is obtained. By discriminating whether the user's handwritten signature is a true sample through the model, the user can log in to their account by handwritten signature on the touch screen and cannot log in to other people's accounts by forging and imitating the handwritten signature.

[0090] In summary, the present invention adopts the method of human-computer interaction and the user's handwritten signature samples collected on the touch screen, selects different classifiers for classification according to different features. Since the information provided by the image features, trajectory features, and user personality features is different, the diversity of different features can be used to improve the discrimination accuracy. At the same time, the user's personality can be predicted based on the handwritten signature trajectory information of the user, personalizing the user signature. The support vector machine classifier is supervised by training with data with positive and negative sample labels and personality labels, and a verification accuracy rate of 96% can be achieved, so as to achieve the purpose of correctly verifying whether the user's handwritten signature is a real user signature.

[0091] Although specific embodiments and drawings of the present invention are disclosed for illustrative purposes, the purpose is to help understand the content of the present invention and implement it accordingly. However, those skilled in the art can understand that various substitutions, transformations, and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. The present invention should not be limited to the content disclosed in the best embodiments and drawings of this specification, and the scope of protection required by the present invention is subject to the scope defined by the claims.

Claims

1. A method for verifying handwritten signatures assisted by personality individuation, the steps of which include: Constructing a list of handwritten signatures, the list of handwritten signatures including: the handwritten signature and personality type of the target user; For the handwritten signature to be verified of the target user, respectively obtaining a signature image and a signature trajectory; Performing image classification based on the signature image and combining with the handwritten signature to obtain a first prediction result; Calculating the distance between the signature trajectory and the target signature trajectory in the handwritten signature to obtain a second prediction result; Performing personality type classification based on the signature trajectory and combining with the personality type to obtain a third prediction result; wherein, the performing personality type classification based on the signature trajectory and combining with the personality type to obtain a third prediction result includes: Extracting the trajectory features of the signature trajectory; Inputting the trajectory features into a personality prediction model to obtain a personality type prediction result, wherein the personality prediction model is constructed through the following steps: Quantitatively modeling the personality psychological characteristics of the subjects through the Big Five Personality Inventory to obtain the personality dimensions of the subjects, the personality dimensions including: openness, extraversion, agreeableness, conscientiousness or emotional stability; Taking the trajectory features of the signature trajectories of the subjects as the input, the personality dimensions as the labels, and using the ResNet-18 convolutional neural network for five-classification training to obtain the personality prediction model; Judging whether the personality type prediction result is the label of the personality type to obtain a third prediction result; Performing weighted calculation on the first prediction result, the second prediction result and the third prediction result to obtain the verification result of the handwritten signature to be verified.

2. The method according to claim 1, characterized in that, The performing image classification based on the signature image and combining with the handwritten signature to obtain a first prediction result includes: Performing normalization processing on the signature image; Extracting the features of the normalized image to obtain a feature vector; Inputting the feature vector into an image classification model to obtain the image classification result of the signature image, wherein the image classification model is constructed based on a supervised machine learning algorithm; Judging whether the image classification result is the label of the handwritten signature to obtain a first prediction result.

3. The method according to claim 2, wherein The performing normalization processing on the signature image includes: For the signature image, extracting the minimum bounding rectangle of the signature for position normalization; And, Calculating the invariant rotation angle of orthogonal regression based on the variance of the horizontal coordinates, the variance of the vertical coordinates and the covariance between the horizontal coordinate components and the vertical coordinate components in the signature image for direction normalization; And, Performing normalization processing on the signature in the signature image for size normalization.

4. The method according to claim 2, characterized in that, The extracting the features of the normalized image includes: Performing contour feature extraction on the normalized image, wherein the contour features include: image contour information, image topology information and point offset information of the image; And, Performing extreme value detection and feature point localization on the scale space of the normalized image and clustering the obtained key points to obtain features composed of the number of key points.

5. The method according to claim 2, wherein The supervised machine learning algorithm includes: SVM, neural network or decision tree model.

6. The method according to claim 1, wherein Obtaining a second prediction result by calculating the distance between the signature trajectory and the target signature trajectory in the handwritten signature includes: Respectively extracting the trajectory features of the signature trajectory and the target trajectory features of the target signature trajectory; Based on the signature trajectory and the target signature trajectory, respectively constructing a signature trajectory sequence and a target signature trajectory sequence; According to the trajectory features and the target trajectory features, using the idea of the DTW algorithm and dynamic programming, calculating the length of the shortest matching distance between the signature trajectory sequence and the target signature trajectory sequence; Comparing the length of the shortest matching distance with a threshold to obtain the second prediction result.

7. The method according to claim 6, wherein The calculating the length of the shortest matching distance between the signature trajectory sequence and the target signature trajectory sequence according to the trajectory features and the target trajectory features, using the idea of the DTW algorithm and dynamic programming includes: Calculating a distance matrix for each point between the signature trajectory sequence and the target signature trajectory sequence; Based on the distance matrix, obtaining the shortest matching path, where the sum of the elements on the shortest matching path is the smallest; Obtaining the distance length of the shortest matching path.

8. A storage medium, in which a computer program is stored, wherein, The computer program is configured to execute any one of the methods recited in claims 1-7 when running.

9. An electronic device, characterized in that, Including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the methods recited in claims 1-7.

Images