signature verification

Abstract

Methods, systems, and computer program products for signature verification are provided. Using real signatures, signature verification can be provided against a target signature. A signature verification model pipeline can extract features from both the target and real signatures, encode both, and submit them to a neural network to generate a similarity score, which can be repeated for each real signature. For example, when one or more similar scores exceed a real threshold, the target signature can be classified as real. The signature verification model can be updated or calibrated at any time using new real signatures. The signature verification model can be implemented using multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and / or classification).

Description

Background Technology

[0001] Digital signatures (e.g., made by a finger or stylus applied to a capturing surface) can be collected from people for use in many types of transactions. A user's identity is typically determined based on physical descriptive information (e.g., appearance, identification documents) or possession of personal information such as passwords. Summary of the Invention

[0002] This invention provides a simplified overview of a series of concepts, which are further described below in specific embodiments. This invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0003] Methods, systems, and computer program products for signature verification are provided. Signature verification can be provided against a target signature using (e.g., only) genuine signatures. A signature verification model pipeline can extract features (e.g., signature stroke geometry and time) from both the target and genuine signatures, encode both, and submit them to a neural network to generate a similarity score, which can be repeated for each genuine signature. For example, when one or more similarity scores exceed a genuine threshold, the target signature can be classified as genuine. The signature verification model can be updated or calibrated at any time using new genuine signatures. The signature verification model can be implemented using multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and / or classification).

[0004] Further features and advantages of the invention, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. It should be noted that the invention is not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to those skilled in the art based on the teachings contained herein. Attached Figure Description

[0005] The accompanying drawings, which are incorporated herein and form a part of this specification, illustrate embodiments of the present application and, together with this specification, further serve to explain the principles of these embodiments and enable those skilled in the art to utilize them.

[0006] Figure 1 A block diagram of a signature verification system according to an example embodiment is shown.

[0007] Figure 2 A block diagram of a graphical user interface for a signature verification system according to an example embodiment is shown.

[0008] Figure 3 A block diagram of a signature verification model for a signature verification system according to an example embodiment is shown.

[0009] Figure 4 A block diagram of a signature verification model for a signature verification system according to an example embodiment is shown.

[0010] Figure 5 A block diagram illustrating feature transformations according to an example embodiment is shown.

[0011] Figure 6 A flowchart is shown for a method of using a graphical user interface for a signature verification system, according to an example embodiment.

[0012] Figure 7 A flowchart of a method for training a signature verification model according to an example embodiment is shown.

[0013] Figure 8 A flowchart of a method for verifying a target signature according to an example embodiment is shown.

[0014] Figure 9 A block diagram of an example computing device that can be used to implement the example embodiments is shown.

[0015] The features and advantages of the invention will become more apparent from the detailed description set forth below when understood in conjunction with the accompanying drawings, in which similar reference characters identify corresponding elements throughout. In the drawings, similar reference numerals generally indicate identical, functionally similar, and / or structurally similar elements. The element first appearing in the drawings is indicated by the leftmost digit of the corresponding reference numeral(s). Detailed Implementation

[0016] I. Introduction

[0017] This specification and accompanying drawings disclose one or more embodiments incorporating the features of the invention. The scope of the invention is not limited to the disclosed embodiments. The disclosed embodiments are merely illustrative of the invention, and modifications to the disclosed embodiments are also included in the invention. The embodiments of the invention are defined by the appended claims.

[0018] References to "an embodiment," "embodiment," "example embodiment," etc., in this specification indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, it should be assumed that implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) is within the knowledge of those skilled in the art.

[0019] In this discussion, unless otherwise stated, modifiers such as “substantially” and “approximately” that modify a condition or relational characteristic of one or more features of an exemplary embodiment of this disclosure are understood to mean that the condition or characteristic is defined within an operationally acceptable tolerance for the intended application of that embodiment.

[0020] Many exemplary embodiments are described below. It should be noted that any section headings provided herein are not intended to be limiting. Embodiments are described throughout this document, and embodiments of any type may be included under any section. Furthermore, embodiments disclosed in any section may be combined in any manner with any other embodiments described in the same section and / or different sections.

[0021] II. Example Implementation

[0022] Digital signatures can be verified, for example, to confirm identity. Signatures can vary from one to the next. Intrapersonal signature variations (i.e., variations between a person's own signatures) can be caused, for example, by whether a person is in a hurry, whether they are using a particular instrument, the shape or style of the instrument, the shape, size, and angle of the digital sampling plate, variations in how a person writes their own signature, and so on. Forgery can mimic signature strokes, leaving very few interpersonal variations (i.e., variations between different people writing the same signature) to distinguish it. Signature verification models can be trained on genuine and forged signatures to learn to distinguish them. Other signature verification models can be trained only on genuine signatures. Some models can be calibrated after initial training (e.g., using additional genuine signatures), while others can remain uncalibrated.

[0023] Therefore, methods, systems, and computer program products for signature verification are provided. Signature verification can be provided against a target signature using (e.g., only) genuine signatures. A signature verification model pipeline can extract features (e.g., signature stroke geometry and / or temporal features) from both the target and genuine signatures, encode both, and submit them to a neural network to generate a similarity score, which can be repeated for each genuine signature. For example, when one or more similarity scores exceed a genuine threshold, the target signature can be classified as genuine. The signature verification model can be updated or calibrated at any time using new genuine signatures. The signature verification model can be implemented using multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and / or classification). Signature verification can be implemented using various components and processes in various embodiments. Some examples of many possible examples are shown and described herein.

[0024] For example, Figure 1 A block diagram of a signature verification system according to an example embodiment is shown. Figure 1As shown, system 100 may include a computing device 110, one or more servers 122, and a storage device 118 that can be communicatively coupled, for example, via network 120. The computing device 110 includes a signature verification application 112, which includes a graphical user interface (GUI) and a signature verification model 116. The servers(s) 122 include a signature verification application 124 and a signature verification model 126. Figure 1 Several computing environments are presented among many computing environments in which the topics described herein can be implemented. As indicated by the dashed lines, in various implementations, network 120 and / or (multiple) servers 122 may or may not be part of the verification system, which may, for example, affect the location of signature verification models 116 / 124 and storage device 118.

[0025] A user can be someone who provides a signature for one or more purposes (such as an identifier in a confirmation or authorization process). Example system 100 can be interacted with by multiple users (such as user 1 102 and user 2 104). In various implementations, there can be one or more users (e.g., tens, hundreds, thousands, millions) providing signatures via one or more signature input devices.

[0026] The signature input device 106 can include any input device from a wide variety of settings. The computing device 110 and the signature input device 106 can be implemented in any location (e.g., fixed or mobile) for any purpose, such as any business (e.g., a bank, merchant, law firm), government entity (e.g., a post office), residence (e.g., where a user needs a signature to log in to a computer), and so on. In the example, the computing device 110 and the signature input device 106 can be integrated, for example, into a touchscreen computing device (e.g., a tablet, cellular phone, self-checkout machine, credit card terminal).

[0027] Computing device 110 may include any computing device. Computing device 110 can be, for example, any type of fixed or mobile, wired or wireless computing device, such as a mobile computer or mobile computing device (e.g., Microsoft). Devices, personal digital assistants (PDAs), laptops, notebook computers, such as Apple iPads TMThe computing device 110 may include tablet computers, netbooks, mobile phones, wearable computing devices, or other types of mobile devices, or fixed computing devices such as desktop computers or PCs (personal computers) or servers. The computing device 110 may include one or more applications, operating systems, virtual machines, storage devices, etc., that can be executed, hosted, and / or stored therein or via one or more other (e.g., networked) computing devices. In the example, the computing device 110 may (e.g., via a network) access one or more server computing devices. An example computing device with the example features is presented in... Figure 8 The details are described below.

[0028] Computing device 110 can execute one or more applications, such as signature verification application 112. In this example, signature verification application 112 may be provided by or accessed via server(s) 122, for example, as a web application implemented within a browser application window on computing device 110. Signature verification application 112 may be configured to communicate with one or more input devices (e.g., signature input device 106). Signature input device 106 may provide a display and sensors to receive and sample signature input provided by user 1102, user 2 104, etc. Signature verification application 112 may control signature input device 106 to display, for example, messages, prompts, and / or feedback to the user (e.g., displaying a signature when the user provides a signature). Signature verification application 112 may receive sampled input signature 108 from signature input device 106. Signature verification application 112 may, for example, clear the signature input device after receiving signature 108.

[0029] The signature verification application 112 may include a GUI for various purposes, including allowing user interaction with the signature verification application 112. In the example, a user who receives a signature from the signature input device 112 (e.g., a bank employee, credit card merchant, government employee, or home user) can view the signature, control what the signature verification application 112 does to the signature (e.g., save it as a genuine signature for an authorized or known user, calibrate the model using one or more genuine signatures, test the model by classifying the signature as genuine or forged), view the classification results, calibrate the signature verification model 116, view the target signature and compare the target signature with a genuine signature (e.g., from storage device 118), and so on.

[0030] The signature verification application 112 may include a signature verification model 116. The signature verification model 116 may receive (e.g., signature 108 allegedly from user 1 102) a target signature and multiple known genuine signatures from user 1 (e.g., from storage device 118) (e.g., from signature input device 108 and signature verification application 112). The signature verification model 116 may classify the target signature (e.g., signature 108) as genuine or forged based on its similarity to (or lack thereof) with the multiple genuine signatures. The signature verification model 116 may include a trainable model. The signature verification model 116 may include one or more (e.g., trainable) neural networks. The signature verification model 116 may be trained, for example, using genuine signatures. The signature verification model 116 may be calibrated after training, for example, using additional genuine signatures. The additional signature may be provided, for example, by an authorized user providing an additional signature (e.g., signature 108) via signature input device 106 and commanding the signature verification application 112 to accept the signature (e.g., signature 108) as a genuine signature, and using the additional genuine signature, which can be stored in storage device 118, to calibrate the model.

[0031] Storage device 118 may include one or more storage devices, which may be local and / or remote (e.g., accessed via network 120). Storage device 118 may include, for example, internal or external storage for computing device 110, internal or external storage for server(s)(s), network storage, etc. Storage device 118 may store user information (e.g., authorized user data, real signatures, and / or other identifying information). User information may be stored in storage device 118 in any format (such as encrypted signatures or signature features indexed by the user, etc.). In the example, storage device 118 may store multiple real signatures of multiple (e.g., tens, hundreds, thousands, millions, tens of millions) users. In the example, signature verification applications 112 and / or 124 may access storage device 118 to store and retrieve information related to users (e.g., user 1 102, user 2 104), their signatures (e.g., real signatures of each user), signature verification application 112, signature verification model 116, etc.

[0032] Network 120 may include any of the following: Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), a combination of communication networks such as the Internet, and / or virtual networks. In an example implementation, computing device 110 and servers(s)122 may be communicatively coupled via network 120. Both servers(s)122 and computing device 110 may include at least one network interface that enables communication via network 120. Examples of such network interfaces (wired or wireless) include IEEE 802.11 Wireless LAN (WLAN) interfaces, Global System for Microwave Access Interoperability (Wi-MAX) interfaces, Ethernet interfaces, Universal Serial Bus (USB) interfaces, cellular network interfaces, and Bluetooth interfaces. TM Interfaces, near field communication (NFC) interfaces, etc. Further examples of network interfaces are described below.

[0033] Multiple servers 122 may include one or more servers, such as one or more application servers, database servers, authentication servers, etc. Multiple servers 122 may support interaction with computing device 110 and storage device 118. Multiple servers 122 may provide data (e.g., real signatures of one or more users) and / or programs (e.g., signature verification application 112 and / or signature verification model 116) to computing device 110. Multiple servers 122 may, for example, provide an application programming interface (API) for signature verification application 112 to interact with storage device 130. Multiple servers 122 may manage, process, preserve, and retrieve data in storage device 118. Multiple servers 122 may include, for example, a database engine or management system (DBMS), such as when storage device 118 includes a database. An example of a database server is Microsoft Azure SQL Server.

[0034] Multiple servers 122 may, for example, provide cloud user authorization and / or signature verification services. Multiple servers 122 may utilize multiple signature input devices 106 for multiple (e.g., tens, hundreds, thousands, millions) users to provide signature verification services for multiple (e.g., tens, hundreds, thousands, millions) entities. In the example, multiple servers 122 may execute a signature verification application 112 and / or a signature verification model 124. Multiple servers 122 may, for example, receive a signature 108 from computing device 110 (e.g., via network 120) and allegedly associated with a user. Signature verification application 112 may provide signature 108 (considered the target signature) and a genuine signature from storage device 118 (e.g., for the user allegedly associated with signature 108) to signature verification model 124. Signature verification model 126 may process signature 108 relative to the genuine signature for the alleged user and return a classification, for example, with a confidence level (e.g., genuine or forged), to computing device 110. The signature verification application 112 can display the results on the GUI 114 as a classification, such as to a bank employee, government employee, credit card merchant, or another party who has received the signature 108 and is waiting for the signature to be accepted or rejected.

[0035] Figure 2 A block diagram of a graphical user interface (GUI) 200 for a signature verification system according to an example embodiment is shown. The example GUI 200 illustrates one of many example implementations of a user interface that can be displayed on a computing device (e.g., computing device 110). In the example (e.g., such as...) Figure 2 As shown, GUI 202 may include multiple icons that a user can select to control a signature verification application (e.g., signature verification application 112). The illustrated icons are signature text 204, classification 206, save as real 208, training / calibrating 210, clear 212, and exit 214. Any combination of these icons, alternative icons, and / or other types of user interface controls may exist in the embodiments. The characteristics of GUI 202 are described below.

[0036] The signature text 204 can be selected, for example, to enter text for the name of the user signing. In the example, the name of the user signing can be entered automatically, such as after scanning a credit card or identification document (e.g., a driver's license).

[0037] Category 206 can be selected, for example, to indicate that one or more incoming signatures (e.g., signature 108) should be categorized (e.g., as opposed to being entered as a genuine signature). Category 206 can be selected, for example, to test the signature against a genuine signature. In the example, the categorization can be automated for one or more implementations, such as in response to (e.g., programmatic) uses (e.g., using a signature verification application for credit card authorization).

[0038] The option to save as a genuine 208 can be selected, for example, to indicate that one or more incoming signatures (e.g., signature 108) should be saved as the user's signature. In the example, a bank could operate a signature verification application that instructs a user (e.g., using one or more input devices) to sign his / her name multiple times (e.g., 10 or 20 times) to collect a set of genuine signatures that can be used to verify the target signature at other times and / or in different locations (e.g., for credit card purposes or bank account access).

[0039] The training / calibration model 210 can be selected, for example (e.g., to a signature verification application), to instruct the signature verification model (e.g., signal verification model 116) to be trained or calibrated using all known real signatures of the user. In the example, the trained model can be calibrated using one or more additional real signatures.

[0040] Clear 212 can be selected, for example, to clear the sampled signature display area 216 and / or clear the signature input device (e.g., to prepare for receiving a new signature).

[0041] Exit 214 can be selected, for example, to exit the GUI 202 and / or the signature verification application.

[0042] The signature display area 216 can display, for example, a signature input box. The signature display area 216 can display a sampled image of the most recent signature (e.g., Ross 218). In this example, the signature display area 216 can match what is displayed on the signature input device.

[0043] Figure 3 A block diagram of a signature verification model 308 for a signature verification system according to an example embodiment is shown. Example model 308 illustrates one of many possible implementations. In the example, signature verification model 308 may include examples of signal verification models 116 and / or 126. Figure 3As shown, the signature verification model 308 includes a first feature extractor 310, a second feature extractor 312, a first feature transformer 314, a second feature transformer 316, and a classifier 318. The signature verification model 308 can receive, for example, a target signature 302 and multiple genuine signatures (e.g., genuine signature 1 306, genuine signature 2 304, etc.). The signature verification model 308 can generate a target signature classification 318. The signature verification model 308 can classify the target signature 302 based on multiple genuine signatures (e.g., genuine signature 1 306, genuine signature 2 304, etc.). The classification can include, for example, genuine and forged. The classification can include or be accompanied by similarity scores, thresholds, and / or confidence levels.

[0044] The signature verification model 308 may include a feature extractor 310, a feature transformer 314, and a classifier 318. The signature verification model 308 may include multiple pipelines for processing the target signature and the genuine signature. For example, the target signature 302 may be processed by the feature extractor 310 and the feature transformer 314, while the genuine signature may be processed by the feature extractor 312 and the feature transformer 316 before the classifier 318. In (e.g., alternative) examples, a single pipeline may exist for the target signature. For example, the genuine signature may be stored in a pre-processed format, which may be encrypted before being provided to the classifier 318.

[0045] Feature extractor 310 can extract features from target signature 302. Feature extractor 310 can extract signature data from signature image. Target signature provided to signature verification model 308 may include geometric and temporal information. In the example, target signature provided to signature verification model 308 may have been sampled at fixed time intervals (e.g., by a signature input device or signature verification application). Each signature stroke segment between any consecutive sampled points may have an equivalent time interval. Feature extractor 310 may interpolate a sufficient number of points (e.g., with small time differences between sampled points) to, for example, make the signature path reconstructed from the sampled points smooth. Each sampled point may include two-dimensional (e.g., x, y) coordinates and temporal information. Each segment (including information for two consecutive points) may include multiple features. Feature extractor 310 may extract multiple features from target signature (e.g., by using interpolated points and segments and associated information). In the example, feature extractor 310 may extract seven geometric features and two temporal features. Feature extractor 310 may be trainable.

[0046] Feature extractor 312 extracts features from a real signature. Feature extractor 312 can extract signature data from a signature image. The real signature provided to signature verification model 308 may include geometric and temporal information. In the example, the real signature provided to signature verification model 308 may have been sampled (e.g., by a signature input device or signature verification application) at fixed time intervals. Each signature stroke segment between any consecutive sampled points may have an equivalent time interval. Feature extractor 310 may interpolate a sufficient number of points (e.g., with small time differences between sampled points) to, for example, make the signature path reconstructed from the sampled points smooth. Each sampled point may include two-dimensional (e.g., x, y) coordinates and temporal information. Each segment (including information for two consecutive points) may include multiple features. Feature extractor 312 may (e.g., by using interpolated points and segments and associated information) extract multiple features from the real signature. In the example, feature extractor 312 may extract seven geometric features and two temporal features. Feature extractor 312 may be trainable.

[0047] Feature transformer 314 can transform the features extracted from target signature 302 for use in classifier 318. Classifier 318 can be configured for features in a specific format for the target signature. Feature transformer 314 can, for example, transform the output of feature extractor 310 into a feature vector expected by classifier 318. Feature transformer 314 can be trainable. In an example, feature transformer 314 can transform the output of feature extractor 310 from a 3D tensor into an encoded matrix and (e.g., then) an encoded vector to provide input to classifier 318.

[0048] Feature transformer 316 can transform features extracted from real signatures (e.g., 306, 304) for use in classifier 318. Classifier 318 can be configured to use features in a specific format for real signatures. Feature transformer 316 can, for example, transform the output of feature extractor 312 into a feature vector expected by classifier 318. Feature transformer 316 can be trainable. In an example, feature transformer 316 can transform the output of feature extractor 312 from a 3D tensor into an encoded matrix and (e.g., then) an encoded vector to provide input to classifier 318.

[0049] Classifier 318 can classify target signature 302 as genuine or counterfeit based on processing the encoded target feature vector from feature transformer 314 and the encoded real feature vector from feature transformer 316. Classifier 318 can generate a similarity score for each encoded real feature vector from feature transformer 316 compared to the encoded target feature vector from feature transformer 314. Classifier 318 can generate target signature classification 320, for example, by comparing each similarity score to a threshold. In the example, a similarity score above the threshold can cause target signature 302 to be classified as genuine. Feature transformer 316 can be trainable.

[0050] Figure 4 A block diagram of a signature verification model 448 for a signature verification system according to an example embodiment is shown. Example model 448 illustrates one of many possible implementations. In the example, signature verification model 448 may include examples of signal verification models 116, 126, and / or 308. Signature verification model 448 may receive, for example, a target signature 402 and multiple genuine signatures (e.g., genuine signature 1 406, genuine signature 2 404, etc.). Signature verification model 448 may generate a target signature classification 436. Signature verification model 448 may classify the target signature 402 based on the multiple genuine signatures (e.g., genuine signature 1 406, genuine signature 2 404, etc.). The classification may include, for example, genuine and forged. The classification may include or be accompanied by similarity scores, thresholds, and / or confidence levels.

[0051] The signature verification model 448 may include a target feature extractor (TFE) 408, a ground truth feature extractor (GFE) 414, a target feature transformer (TFT) 420, a ground truth feature transformer (GFT) 426, and a classifier 438. The signature verification model 448 may include multiple pipelines for processing the target signature and the ground truth signature. For example, the target signature 402 may be processed by TFE 408 and TFT 420, while the ground truth signature may be processed by GFE 414 and GFT 426 (e.g., before being classified by classifier 438). In an alternative embodiment, a single pipeline may exist for the target signature. For example, the ground truth signature may be stored in a pre-processed format, which may be protected (e.g., by encryption) before being provided to classifier 438.

[0052] The Target Feature Extractor (TFE) 408 can extract features from the target signature 402. The TFE 408 can extract signature data from the signature image. The target signature provided to the signature verification model 448 can include geometric and temporal information. In the example, the target signature provided to the signature verification model 448 may have been sampled (e.g., by a signature input device or signature verification application) at fixed time intervals. Each signature stroke segment between any consecutive sampled points can have an equivalent time interval. The TFE 408 can interpolate a sufficient number of points (e.g., with small time differences between sampled points) to, for example, make the signature path reconstructed from the sampled points smooth. Each sampled point can include two-dimensional (e.g., x, y) coordinates and temporal information. Each segment (including information for two consecutive points) can include multiple features. The TFE 408 can extract multiple features from the target signature (e.g., by using interpolated points and segments and associated information). In the example, the TFE 408 can extract seven geometric features and two temporal features. In the example, the background pixels can be set to zero. The pixels corresponding to stroke points can be normalized to [-1, 1]. TFE 408 can be trainable.

[0053] The Genuine Feature Extractor (GFE) 414 extracts features from a genuine signature (e.g., 406, 404, etc.). GFE 414 can extract signature data from a signature image. The genuine signature provided to the signature verification model 448 may include geometric and temporal information. In the example, the genuine signature provided to the signature verification model 448 may have been sampled (e.g., by a signature input device or signature verification application) at fixed time intervals. Each signature stroke segment between any consecutive sample points may have an equivalent time interval. GFE 414 may interpolate a sufficient number of points (e.g., with small time differences between sample points) to, for example, make the signature path reconstructed from the sample points smooth. Each sample point may include two-dimensional (e.g., x, y) coordinates and temporal information. Each segment (including information for two consecutive points) may include multiple features. GFE 414 can extract multiple features from the genuine signature (e.g., by using interpolated points and segments and associated information). In the example, GFE 414 may extract seven geometric features and two temporal features. In the example, background pixels may be set to zero. The pixels corresponding to stroke points can be normalized to [-1, 1]. GFE 414 can be trained.

[0054] In the example (e.g., as...) Figure 4As shown, TFE 408 may include, for example, a Temporally Enhanced Path Signature Feature (TEPSF) extractor 410, while GFE 414 may include a TEPSF extractor 416. TEPSF extractor 410 may operate similarly (e.g., identically) to TEPSF extractor 416. TEPSF extractors 410 and 416 may include: an improved PSF extractor configured to interpolate a sufficient number of stroke points to obtain a smooth path, normalize stroke pixels and background, and generate temporal features in addition to geometric and ordering features.

[0055] TEPSF extractors 410 and 416 can extract features from the signature, for example, by extracting signature stroke paths / segments and information associated with consecutive stroke points. In the example, the signature image can be stored (e.g., cached) as a digital link. The cached signature image can provide a sequence or set of strokes, each with a series of stroke points and paths or segments between consecutive stroke points.

[0056] TEPSF extractors 410 and 416 can interpolate a sufficient number of points (e.g., with small time differences between sampling points) to, for example, make the signature path reconstructed from the sampling points smooth. In the example, background pixels can be set to zero. Pixels corresponding to stroke points can be normalized to [-1, 1]. Features can include geometric position, order, and timing information for each segment in one or more signature strokes.

[0057] TEPSF extractors 410 and 416 can, for example, encode each segment into multiple (e.g., nine) 2D matrices. Each matrix can represent a geometric or temporal attribute of the segment. In the example, seven matrices can represent geometric attributes or features for a stroke point or segment, while two matrices can represent temporal attributes or features for a stroke point or segment.

[0058] The signature stroke S can be represented by consecutive dots or segments of consecutive dots, for example, as provided by Equation 1:

[0059] S={(x i y i , t i Equation 1: |i=1,2,...,m}

[0060] Where x i y i Let be the geometric coordinates of the i-th stroke point, and t i This is the timestamp for each i-th stroke point, where i ranges from 1 to m. The stroke path segment (connecting two consecutive points) can be given by Equation 2:

[0061] s i，i+1 ={(x i y i ), (x i+1 y i+1 Equation 2

[0062] Each stroke path segment between consecutive points can include, for example, a geometric signature feature provided by Equation 3 (e.g., with stroke point ordering):

[0063]

[0064]

[0065]

[0066]

[0067] in This represents the Kronecker matrix product. Each stroke path segment may (e.g., also) include, for example, a temporal signature feature provided by Equation 4:

[0068]

[0069] The scalar term can be based on a set of real signatures used to classify the target signature. In the example, the scalar could include, for example, 1 / 1000 (e.g., for real signatures in the Signature Verification Contest (SVC) 2004, which includes a signature database containing 100 signature sets, each with 20 real signatures and 20 skilled forgeries).

[0070] Each of the nine 2D matrices (or nine second-order tensors) can represent the geometric properties of the original stroke points or segments in the signature. The nine matrices representing the nine channels for each stroke point or segment can be stacked into a 3D tensor, for example, as provided in Equation 5:

[0071]

[0072] Where Ws can include the width of each matrix, 128 can include the height of each matrix, and 9 represents nine channels or features. The width can be variable, as indicated by Ws. The height and width of various signatures can vary. One variable can be fixed, while others can scale to accommodate signatures of different sizes.

[0073] The Temporally Enhanced Path Signature Feature (TEPSF) extractor 410 can output a 3D tensor (e.g., as described by Equation 5) of a given target signature 402 as extracted target features. The TEPSF extractor 416 can output a 3D tensor (e.g., as described by Equation 5) for each real signature (e.g., real signature 1406, real signature 2404, etc.) as extracted real features. The extracted target features 412 can be provided to a Target Feature Transformer (TFT) 420 for use in multiple transformations. The extracted real features 418 can be provided to a Genuine Feature Transformer (GFT) 426 for use in multiple transformations.

[0074] TFT 420 and GFT 426 can (e.g., both) provide transformations necessary for the configuration compatibility and / or performance of classifier 438. TFE 408 and GFE 414 can be configured to generate specific outputs for extracted target features 412 and extracted true features 418. Classifier 438 can be configured for specific inputs (e.g., encoded target features 432 and encoded true features 446), which may be incompatible with extracted target features 412 and extracted true features 418. Therefore, TFT 420 and GFT 426 can provide necessary information for one or more uses, such as, but not limited to, compatibility, performance optimization, etc. It should be clear that transformations may or may not be required in various implementations, and (e.g., when required) can vary between implementations based on the feature extraction inputs and classification input configurations selected for various implementations.

[0075] In the example implementation (e.g., as Figure 4 As shown, TFT 420 may include, for example, an encoded convolutional neural network (ECNN) 422 and a long short-term memory (LSTM) 424. GFT 426 may include ECNN 428 and LSTM 430.

[0076] ECNN 422 and LSTM 424 can (e.g., both) be trainable neural networks. For example, ECNN 422 and LSTM 424 can (e.g., both) have trainable weights. Similarly, ECNN 428 and LSTM 430 can (e.g., both) be trainable neural networks. For example, ECNN 428 and LSTM 430 can (e.g., both) have trainable weights.

[0077] ECNN 422 and ECNN 428 can perform similar (e.g., the same) functions. Therefore, they can be described together. Similarly, LSTM 424 and LSTM 430 can perform similar (e.g., the same) functions. Therefore, they can be described together.

[0078] ECNN 422 and LSTM 424 can (e.g., collectively) transform target features 412 extracted from 3D tensors into encoded target features 432 (e.g., an encoded target feature vector represented therefrom). Similarly, ECNN 428 and LSTM 430 can (e.g., collectively) transform ground truth features 418 extracted from 3D tensors into encoded ground truth features 446 (e.g., an encoded ground truth feature vector represented therefrom). Transformation broadly refers to any processing or manipulation of existing data, creating new data from existing data, and / or combinations thereof.

[0079] In the example (e.g., as...) Figure 4 As shown, ECNN 422 and ECNN 428 can transform and encode the extracted target features 412 and the extracted ground truth features 418, respectively. ECNN 422 and ECNN 428 can (in the first transformation step) for example, transform and encode the 3D tensor represented by the extracted target features 412 and the extracted ground truth features 418 into an encoded 3D tensor, for example, as provided by Equation 6:

[0080]

[0081] Where T encode This can represent the output 3D tensor after CNN extraction of the input tensor Ws*128*9, where Ws / 8 can include the (e.g., variable) width of each matrix, 1 can include the height of each matrix, and 256 can represent the number of channels.

[0082] ECNN 422 and ECNN 428 can (e.g., in the second transformation step) transform the encoded 3D tensor in Equation 6 into an encoded 2D matrix, for example, as provided by Equation 7:

[0083]

[0084] The 2D encoded matrix Mencode can have, for example, 256 (e.g., fixed) rows and Ws / 8 (e.g., variable) columns.

[0085] The 3D tensor outputs of TEPSF extractors 410 and 416 can be encoded as 2D matrices, for example, because the task involves processing sequential data. Neural networks can require matrices instead of 3D tensors as input.

[0086] In the example (e.g., as...) Figure 4As shown, LSTM 424 and LSTM 430 can transform each of the target encoded 2D matrix and the real encoded 2D matrix (generated by ECNN422 and ECNN 428, respectively) into encoded vectors represented by the encoded target features 432 and the encoded real features 446, respectively.

[0087] Figure 5 A block diagram illustrating a feature transformation implemented by an LSTM neural network 500 according to an example embodiment is shown. The example LSTM neural network 500 may include examples of LSTM 424 and LSTM 430. The example LSTM neural network 500 (e.g., at a high level) shows that a 2D target feature matrix generated by ECNN 422 and a 2D ground truth feature matrix generated by ECNN 428 can be transformed into an nth encoded target feature vector (represented by encoded target features 432) and an nth encoded ground truth feature vector (represented by encoded ground truth features 446) to be provided as input to classifier 438.

[0088] In the example implementation (e.g., referencing Equation 7), each column of the encoded 2D target feature matrix (generated by ECNN422) and the encoded 2D ground truth feature matrix (generated by ECNN428) can represent a timestamp of the input. As indicated by Equation 7, in this example, there can be, for example, Ws / 8 columns in each matrix.

[0089] like Figure 5 As shown, each column of the 2D matrix can be input into the corresponding LSTM unit. For example, the first matrix column 514 can be input into the first LSTM unit 508, which generates the first encoded vector 502. The second matrix column 516 can be input into the second LSTM unit 510, which generates the second encoded vector 504. This continues up to the nth matrix column 518, which can be input into the nth LSTM unit 518, which generates the nth encoded vector 506. Therefore, LSTM 424 and LSTM 430 can, for example, transform the columns of the encoded 2D matrix in Equation 7 into a series of encoded vectors, as provided by Equation 8:

[0090] M encode [:,1] T ∈R 1×256 →v1∈R 1×512 Formula 8

[0091] M encode [:,2] T ∈R 1×256 →v2∈R 1×512

[0092]

[0093] A series of Ws / 8LSTM cells can exist, for example. Figure 5 As indicated in the example where the number of LSTM units matches the number of columns in a 2D matrix, n units can be equal to Ws / 8 columns. Figure 5 As indicated, each LSTM unit can generate an encoded vector that depends on the previous LSTM unit. Each LSTM unit can provide information (e.g., the trajectory of previous stroke points) to subsequent LSTM units. The nth encoded target feature vector and the nth encoded ground truth feature vector can be provided as input to the classifier 438. These vectors can be of the same size. They can be concatenated together as a single long vector.

[0094] Classifier 438 can classify target signature 402 as genuine or counterfeit based on the similarity between the encoded target feature 432 generated by TFT 420 and the encoded genuine feature 446 generated by GFT 426 for each of the multiple genuine signatures. Figure 4 and Figure 5 In the ongoing example presented, the encoded target feature 432 may include the nth encoded vector 506 generated by LSTM 424, while the encoded ground feature 446 may include the nth encoded vector 506 generated by LSTM 430.

[0095] The classifier 438 may include, for example, a feedforward neural network (FNN) 440 and a comparator 442. The FNN 440 can generate a set of similarity scores, where each score is based on the similarity between a vector including encoded target features 432 and a vector including encoded real features 446. Similarity scores can be generated for each of a plurality of real signature vectors and the target signature vector. Given a signature to be verified and N real signatures, N similarities can exist. The N similarity scores can be represented as S1, S2, S3, ..., S... N In the example, the similarity score generated by FNN 440 (e.g., each) can be in the range of zero to one, for example, similarity ∈ [0, 1].

[0096] Comparator 442 can, for example, be configured by comparing each similarity score S1, S2, S3, ..., S N The target signature 402 is classified by comparing it with one or more (e.g., configurable) thresholds (e.g., threshold 444). Classification can be based on assigning a similarity score S1, S2, S3, ..., S... to each similarity score. NCompare with a threshold T (e.g., threshold 444). In the example, for instance, when at least one similarity score S1, S2, S3, ..., S... N When the threshold T is met or exceeded, the target signature 402 can be classified as genuine. For example, when the similarity scores S1, S2, S3, ..., S... N For each value below the threshold T, the target signature 402 can be classified as forged. In the example, the true classification can be given by Equation 9:

[0097] max{S1, S2, S3, ..., S...} N}≥T Equation 9

[0098] In the example, the threshold T can be set at 50% or 0.5. Different thresholds can be associated with different real signatures in the example.

[0099] FNN 440 can be a trainable neural network. For example, FNN 440 can (e.g., all) have trainable weights. The signature verification model 448 can include multiple trainable neural networks (e.g., ECNN 422, ECNN 428, LSTM 424, LSTM 430, FNN 440). The signature verification model 448 can be trained, for example, using real signatures. The signature verification model 448 can be calibrated, for example, after training with additional real signatures.

[0100] In the example, multiple neural networks (e.g., ECNN 422, ECNN 428, LSTM 424, LSTM 430, FNN440) can be trained, for example, in multiple iterations. At each iteration of the training, optimality can be computed for each neural network. Optimality for each neural network can include, for example, the norm of the gradient of each of the multiple trainable neural networks used for that training iteration. The trainable neural network with the highest computed optimality among the multiple neural networks used for that training iteration can be selected. The selected neural network can be updated for that training iteration, for example, while keeping other neural networks unchanged. Additional iterations can select one or more other neural networks for updates, for example, while keeping one or more other neural networks unchanged. This type of training can prevent training variations that negatively impact performance by simultaneously changing multiple (e.g., all) trainable neural networks.

[0101] The signature verification model 448 can be used to process signatures with and without timestamped stroke points. For example, TEPSF extractors 410 and 416 can (e.g., also) extract features from signatures stored as images (e.g., without timestamped stroke points). In the example, the image (e.g., an RGB image) can (e.g., essentially) be a 3D tensor (e.g., width * height * 3). The outputs of TEPSF extractors 410 and 416 can include a 3D tensor (e.g., width * 128 * 9). The image signature can, for example, be rescaled to a tensor with a height of 128 (e.g., one). The (e.g., first) convolutional layer of a CNN (e.g., ECNN 422, 428) can, for example, resize the input tensor to have three (e.g., instead of nine) channels.

[0102] Signature verification models (e.g., signature verification model 448) can be implemented with or without neural networks. Signature verification models can be implemented using different types of neural networks besides those mentioned in the examples (e.g., ECNN, LSTM, and FNN). Signature verification models can include, for example, any model capable of analyzing time series data (e.g., Hidden Markov Models).

[0103] In the examples, TFT 420 and GFT 426 can have configurable architectures and / or architectures that can be configured. Table 1 provides example architectures and configurations for TFT 420 and GFT 426, each including a CNN-LSTM model. Table 1 presents the input shape in the form of (width, height, channels), the kernel shape in the form of (number of kernels, mask, stride), and the output shape in the form of (width, height, channels). The stride can be, for example, a single number of width and height strides, or a tuple (sW, sH) representing the width and height strides separately.

[0104] Table 1 – Example Architectures and Configurations for TFT and GFT

[0105]

[0106]

[0107] FNN 440 can include multiple fully connected (FC) layers (e.g., FC1, FC2, FC3, FC4). ECNN 422 and 428 can encode PSF tensors from 3D tensors into 2D matrices. Recurrent Neural Networks (RNNs) such as LSTM 424 and 430 can utilize each column or row of a 2D matrix as an input vector for an RNN (e.g., LSTM) unit. The output vector of the last RNN unit (e.g., ...) Figure 4The Nth encoded vector value (R1x512) shown can be used as (e.g., partially) input to the FNN 440, which can be formed by stacking several (e.g., four) fully connected (FC) layers. The FNN 440 computes the similarity between two signatures. The input size of the first FC layer (FC1) can be (e.g., as shown in the example in Table 1) for example, 512*2 = 1024. The output of the last FC layer (FC4) can be the similarity score between the two input signatures.

[0108] Implementations are not limited to the examples shown. Any number of computing devices and / or servers (including, but not limited to, machines and / or virtual machines) can be coupled in any way across any type of computing environment. For example, one or more of the computing devices, servers, or storage components can be co-located, located away from each other, combined or integrated on one or more physical or virtual machines, or distributed across one or more physical or virtual machines. References Figure 1-5 The examples shown and discussed can be, for example, based on Figure 6-8 The example methods presented are for operation.

[0109] Implementation examples can also be set up in processes or methods. For example, Figure 6 A flowchart of a method 600 for using a graphical user interface for a signature verification system, according to an example embodiment, is shown. The embodiments and other embodiments disclosed herein can be operated according to example method 600. Method 600 includes steps 602-606. However, other embodiments can be operated according to other methods. Other structural and operational embodiments will be apparent to those skilled in the art based on the foregoing discussion of the embodiments. The order of steps is not required unless explicitly indicated or inherently required. Implementation of the method embodiments is not required. Figure 6 All steps are shown in the diagram. Figure 6 This is just one of many possible embodiments. Embodiments can implement fewer, more, or different steps.

[0110] Method 600 includes step 602. In step 602, the signature verification application can receive a digital signature when an authorized user logs in to use the signature application. For example, as Figure 1 As shown, the signature verification application 112 executed by the computing device 110 can receive a signature from the user 1102 via the signature input device 106.

[0111] In step 604, a menu selection may be received to save the received signature as a real signature, classify the signature, calibrate the model, clear the signature, or exit the application. For example, as Figure 2As shown, the user of the signature verification application 112 can interact with the GUI 114 / 202 to save the received signature as a real signature 208, classify the signature 206, calibrate the model 210, clear the signature 212, or exit the application 214.

[0112] In step 606, the received signature may, for example, be saved as real or categorized (e.g., real or fake) in response to a menu selection, the model may be calibrated, the signature may be cleared, or the application may be closed / exited. For example, as Figure 1-5 As shown, application 112 can respond to choices made by the user of signature verification application 112 by saving the signature as real, classifying the signature, calibrating the signature verification model 116, clearing the signature from the signature display area 216 and / or the signature input device 106, or exiting the signature verification application 112.

[0113] Figure 7 A flowchart of a method 700 for training a signature verification model according to an example embodiment is shown. The embodiments disclosed herein and other embodiments can be operated according to example method 700. Method 700 includes steps 702-708. However, other embodiments can be operated according to other methods. Other structural and operational embodiments will be apparent to those skilled in the art based on the foregoing discussion of the embodiments. The order of steps is not required unless explicitly indicated or inherently required. Implementation of the method embodiments is not required. Figure 7 All steps are shown in the diagram. Figure 7 This is just one of many possible embodiments. Embodiments may implement fewer, more, or different steps.

[0114] Method 700 includes step 702. In step 702, a signature verification model (e.g., including multiple trainable neural networks) may be trained or calibrated, for example, using multiple real signatures. Figure 1 As shown, signature verification model 116 (e.g., including such as Figure 4 The multiple neural networks shown (ECNN 422, 428, LSTM 424, 430 and FNN 430) can be trained using real signatures in storage device 118.

[0115] In step 704, for each iteration of training, optimality (e.g., including the norm of the gradient) can be computed for each of the multiple trainable neural networks. For example, as Figure 4 As shown, for each iteration of training, optimality (e.g., including the norm of the gradient) can be computed for each of ECNN 422, 428, LSTM 424, 430 and FNN 430.

[0116] In step 706, a trainable neural network with the highest computational optimality can be used for each iteration of training. For example, as Figure 4 As shown, one or more of the neural networks ECNN 422, 428, LSTM424, 430 and FNN 430 that have the highest optimality in the training iterations can be selected.

[0117] In step 708, the selected neural network can be updated for this training iteration. For example, as Figure 4 As shown, one or more of the neural networks selected in the training iterations, such as ECNN 422, 428, LSTM 424, 430 and FNN 430, can be updated.

[0118] Figure 8 A flowchart of a method 800 for verifying a target signature according to an example embodiment is shown. The embodiments disclosed herein and other embodiments can be operated according to example method 800. Method 800 includes steps 802-810. However, other embodiments can be operated according to other methods. Other structural and operational embodiments will be apparent to those skilled in the art based on the foregoing discussion of the embodiments. The order of steps is not required unless explicitly indicated or inherently required. Implementation of the method embodiments is not required. Figure 8 All steps are shown in the diagram. Figure 8 This is just one of many possible embodiments. Embodiments may implement fewer, more, or different steps.

[0119] Method 800 includes step 802. In step 802, a target signature asserted to be associated with a user may be received. For example, such as Figure 1 As shown, the signature verification application 112 executed by the computing device 110 can receive a signature 108 (e.g., target signature 302 or 402) from the user 1 102 via the signature input device 106.

[0120] In step 804, target signature features for the target signature can be identified. For example, such as Figure 3 and Figure 4 As shown, feature extractor 310 and feature transformer 314 or TFE 408 and TFT 420 can extract target signature features for target signature 302 or 402.

[0121] In step 806, the similarity between the target signature features and the real signature features can be determined for each of the multiple real signatures. For example, such as Figure 4 As shown, FNN 440 can determine the similarity between encoded target features 432 and encoded true signature features 446.

[0122] In step 808, a set of similarity scores may be determined (e.g., including similarity scores for each of the determined similarities). For example, as Figure 4 As shown, FNN 440 can generate a set of similarity scores, where each score is based on the similarity determined between encoded target feature 432 and encoded real signature feature 446 for one of a plurality of real signatures.

[0123] In step 810, the target signature can be verified (e.g., whether it is genuine or forged) based on the set of similarity scores. For example, as Figure 4 As shown, comparator 442 can compare each of the set of similarity scores 434 with one or more thresholds 444 to verify whether the target signature 402 is genuine or forged.

[0124] III. Example Computing Device Implementation

[0125] As noted herein, the described embodiments, together with any of their modules, components, and / or sub-components (e.g., computing device 110, servers(multiple) 122, signature verification application 112, signature verification application 124, etc.), and the flowcharts / flowcharts described herein (e.g., example methods 500, 600, 700) (including portions thereof) and / or other embodiments, can be implemented in hardware or any combination of hardware and software and / or firmware, including computer program code implemented as being configured to be executed in one or more processors and stored in a computer-readable storage medium, or implemented as hardware logic / circuit, such as being implemented together in a system-on-a-chip (SoC), field-programmable gate array (FPGA), and / or application-specific integrated circuit (ASIC). An SoC may include an integrated circuit chip that performs its functions, the integrated circuit chip including one or more of the following: a processor (e.g., a microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or additional circuitry and / or embedded firmware.

[0126] Figure 9 An exemplary implementation of a computing device 900 in which example embodiments may be implemented is shown. Consistent with all other descriptions provided herein, the description of the computing device 900 is a non-limiting example for illustrative purposes. Example embodiments may be implemented in other types of computer systems, as will be known to those skilled in the art(s) related to the subject.

[0127] like Figure 9As shown, the computing device 900 includes one or more processors (referred to as processor circuitry 902), system memory 904, and a bus 906 coupling various system components, including system memory 904, to processor circuitry 902. Processor circuitry 902 is an electrical and / or optical circuit implemented in one or more physical hardware circuitry devices and / or integrated circuit devices (semiconductor chip or die) as a central processing unit (CPU), microcontroller, microprocessor, and / or other physical hardware processor circuitry. Processor circuitry 902 can execute program code stored in a computer-readable medium, such as operating system 930, application program 932, other programs 934, etc. Bus 906 represents any one or more of several types of bus architectures, including memory bus or memory controller, peripheral bus, accelerated graphics port and processor, or a local bus using any of various bus architectures. System memory 904 includes read-only memory (ROM) 908 and random access memory (RAM) 910. Basic input / output system 912 (BIOS) is stored in ROM 908.

[0128] The computing device 900 also includes one or more of the following drivers: a hard disk driver 914 for reading from and writing to a hard disk, a disk driver 916 for reading from or writing to a removable disk 918, and an optical disc driver 920 for reading from or writing to a removable optical disc 922 (such as a CD-ROM, DVD-ROM, or other optical media). The hard disk driver 914, disk driver 916, and optical disc driver 920 are connected to the bus 906 via a hard disk driver interface 924, a disk driver interface 926, and an optical disc driver interface 928, respectively. The drivers and their associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for a computer. Although hard disks, removable disks, and removable optical discs are described, other types of hardware-based computer-readable storage media can be used to store data, such as flash memory cards, digital video discs, RAM, ROM, and other hardware storage media.

[0129] Many program modules may be stored on a hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include an operating system 930, one or more application programs 932, other programs 934, and program data 936. Application programs 932 or other programs 934 may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the example embodiments described herein.

[0130] Users can input commands and information into computing device 900 through input devices such as keyboard 938 and pointing device 940. Other input devices (not shown) may include microphone, joystick, game controller, satellite dish, scanner, touch screen and / or touchpad, voice recognition system for receiving voice input, gesture recognition system for receiving gesture input, and so on. These and other input devices are typically connected to processor circuitry 902 via serial port interface 942 coupled to bus 906, but may be connected via other interfaces such as parallel port, game port, or Universal Serial Bus (USB).

[0131] Display screen 944 is also connected to bus 906 via an interface such as video adapter 946. Display screen 944 may be external to or incorporated into computing device 900. Display screen 944 may display information and serve as a user interface for receiving user commands and / or other information (e.g., via touch, finger gestures, virtual keyboard, etc.). In addition to display screen 944, computing device 900 may include other peripheral output devices (not shown) such as speakers and printers.

[0132] Computing device 900 is connected to network 948 (e.g., the Internet) via an adapter or network interface 950, modem 952, or other means for establishing communication over the network. Modem 952 (which may be internal or external) can be connected via, for example... Figure 9 The serial port interface 942 shown is connected to bus 906, or another interface type including a parallel interface can be used to connect to bus 906.

[0133] As used herein, the terms “computer program medium,” “computer-readable medium,” and “computer-readable storage medium” are used to refer to physical hardware media, such as hard disks associated with hard disk drives 914, removable disks 918, and removable optical disks 922; other physical hardware media, such as RAM, ROM, flash memory cards, digital video disks, compressed disks, MEM, nanotechnology-based storage devices, and other types of physical / tangible hardware storage media. Such computer-readable storage media are distinct from and do not overlap with communication data (excluding communication media). Communication media embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves. The term “modulated data signal” means a signal whose characteristics are set or altered such that information is encoded in the signal. By way of example, and not limitation, communication media include wireless media such as acoustic, RF, infrared, and other wireless media, as well as wired media. The exemplary embodiments also relate to such communication media that are distinct from and do not overlap with embodiments relating to computer-readable storage media.

[0134] As noted above, computer programs and modules (including application program 932 and other programs 934) can be stored on a hard disk, magnetic disk, optical disk, ROM, RAM, or other hardware storage medium. Such computer programs can also be received via network interface 950, serial port interface 942, or any other interface type. When executed or loaded by an application, such computer programs enable computing device 900 to implement the features of the exemplary embodiments described herein. Therefore, such computer programs represent the controller of computing device 900.

[0135] The example embodiments also relate to computer program products that include computer code or instructions stored on any computer-readable medium. Such computer program products include hard disk drives, optical disk drives, storage device packages, memory sticks, memory cards, and other types of physical storage hardware.

[0136] IV. Example Implementation

[0137] Methods, systems, and computer program products for signature verification are provided. Signature verification against a target signature can be provided using (e.g., only) genuine signatures. A signature verification model pipeline can extract features (e.g., signature stroke geometry and time) from both the target and genuine signatures, encode both, and feed them into a neural network to generate a similarity score, which can be repeated for each genuine signature. For example, when one or more similarity scores exceed a genuine threshold, the target signature can be classified as genuine. The signature verification model can be updated or calibrated at any time using new genuine signatures. The signature verification model can be implemented using multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and / or classification).

[0138] In the example, a method (e.g., for verifying a target signature in a signature verification model pipeline) may include, for example: receiving a target signature asserted to be associated with a user by a computing device to generate a plurality of determined similarities; determining the similarity between the target signature and each of a plurality of real signatures associated with the user; generating a set of similarity scores, which includes a similarity score for each of the determined similarities; and verifying the target signature based on the set of similarity scores.

[0139] In the example, verifying the target signature may include, for example, classifying the target signature as either genuine or forged.

[0140] In the example, for instance, a target signature can be classified as real when at least one of a set of similarity scores exceeds the true classification threshold.

[0141] In the example, determining the similarity between the target signature and each of the multiple real signatures may include, for example: identifying the target signature features for the target signature; identifying the real signature features for each of the multiple real signatures; and determining the similarity between the target signature features and the real signature features for each of the multiple real signatures.

[0142] In the example, the target signature feature may include a time feature based on a timestamp associated with a sampling point on the target signature stroke, and the real signature feature includes a time feature based on a timestamp associated with a sampling point on the real signature stroke.

[0143] In the example, the method may further include, for example: encoding target signature features into encoded target features; encoding real signature features into encoded real features for each of a plurality of real signatures; submitting the encoded real features and encoded target features for each of the plurality of real signatures as input to a neural network; and generating a similarity score for each submission by the neural network, which together include a set of similarity scores.

[0144] In the example, the method may also include training the neural network by, for example, by: (e.g., for each of multiple iterations of training), calculating optimality, which includes the norm of the gradient of each of the multiple trainable neural networks used for iterative training; selecting the trainable neural network with the highest calculated optimality for iterative training; and updating the selected neural network for training iterations.

[0145] In the example, the method may also include, for example, receiving an additional real signature associated with the user; and using the additional real signature to calibrate the method.

[0146] In the example, a signature verification system may include, for example, one or more processors; and one or more memory devices storing program code configured to be executed by the one or more processors. The program code may include, for example, a model configured to: receive a target signature asserted to be associated with a user; generate a set of similarity scores, including a similarity score for each of a plurality of genuine signatures relative to the target signature; and verify the target signature based on the set of similarity scores.

[0147] In the example, for instance, a target signature can be classified as real when one or more of a set of similarity scores exceed the true classification threshold.

[0148] In the example, the model may include, for example, a feature extractor configured to: identify target signature features for the target signature; and identify real signature features for each of the multiple real signatures.

[0149] In the example, the target signature feature may include a time feature based on a timestamp associated with a sampling point on the target signature stroke, and the real signature feature may include a time feature based on a timestamp associated with a sampling point on the real signature stroke.

[0150] In the example, the model may also include, for example, an encoder configured to: encode the target signature features into encoded target features; and for each of the plurality of real signatures, encode the real signature features into encoded real features.

[0151] In the example, the model may also include, for example, a neural network configured to: receive encoded target features and encoded real features for each of a plurality of real signatures; and generate a set of similarity scores including a similarity score for each of the plurality of real signatures represented by the encoded real features relative to a target signature represented by the encoded target features.

[0152] In the example, the model may also include, for example, a trainer configured to train the neural network over multiple iterations, including: for each iteration of training: calculating optimality, which includes the norm of the gradient of each of the plurality of trainable neural networks used for the training iteration; selecting the trainable neural network with the highest calculated optimality for the training iteration; and updating the selected neural network for the training iteration.

[0153] In the example, a computer-readable storage medium having program instructions recorded thereon executes a method when performed by processing circuitry, the method comprising, for example: providing a target signature asserted to be associated with a user and a plurality of genuine signatures associated with the user to a model; generating a set of similarity scores from the model, the set of similarity scores including a similarity score for each of the plurality of genuine signatures relative to the target signature; and classifying the target signature as either genuine or forged based on the set of similarity scores.

[0154] In the example, for instance, a target signature can be classified as real when one or more of the similarity scores in the group exceed the true classification threshold.

[0155] In the example, the method may also include, for example: identifying target signature features for the target signature by the model; identifying real signature features for each of the multiple real signatures by the model among the multiple real signatures; and generating the set of similarity scores by the model based on the target signature features and the real signature features.

[0156] In the example, the target signature feature may include a time feature based on a timestamp associated with a sampling point on the target signature stroke, and the real signature feature includes a time feature based on a timestamp associated with a sampling point on the real signature stroke.

[0157] In the example, the method may also include, for example: encoding the target signature features into encoded target features by the model; encoding the real signature features for each of the multiple real signatures into encoded real features by the model; and generating the set of similarity scores by the model based on the encoded real features and the encoded target features.

[0158] In the example, the model can use a trainable neural network to generate the similarity scores for the group.

[0159] V. Conclusion

[0160] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitation. Those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. Therefore, the breadth and scope of the invention should not be limited by any of the exemplary embodiments described above, but should be defined only by the appended claims and their equivalents.

Claims

1. A method for verifying a target signature, comprising: for each iteration of a plurality of iterations of training, training a plurality of neural networks using a plurality of real signatures associated with a user by: computing an optimality comprising a norm of a gradient for each neural network of the plurality of neural networks; selecting a neural network of the plurality of neural networks having a highest optimality computed; updating the selected neural network while leaving other neural networks of the plurality of neural networks unchanged; receiving, by a computing device, the target signature asserted to be associated with the user; determining, by a signature verification model comprising the trained plurality of neural networks: a similarity between the target signature and each real signature of the plurality of real signatures to generate a plurality of determined similarities; generating a set of similarity scores comprising a similarity score for each of the determined similarities; and verifying the target signature based on the set of similarity scores.

2. The method of claim 1, wherein the verifying the target signature comprises: classifying the target signature as real or fake.

3. The method of claim 2, wherein the classifying the target signature as real or fake comprises: classifying the target signature as real when at least one score of the set of similarity scores exceeds a real classification threshold.

4. The method of claim 1, wherein the determining a similarity between the target signature and each real signature of a plurality of real signatures associated with the user to generate a plurality of determined similarities comprises: identifying a target signature feature for the target signature; identifying a real signature feature for each real signature of the plurality of real signatures in the plurality of real signatures; and determining a similarity between the target signature feature and the real signature feature for each real signature of the plurality of real signatures.

5. The method of claim 4, wherein the target signature feature comprises a temporal feature based on timestamps associated with sampling points on a target signature stroke; and wherein the real signature feature comprises a temporal feature based on timestamps associated with sampling points on a real signature stroke.

6. The method of claim 5, further comprising: encoding the target signature feature as an encoded target feature; encoding the real signature feature for each real signature of the plurality of real signatures as an encoded real feature; submitting the encoded real feature for each real signature of the plurality of real signatures and the encoded target feature as input to a neural network; and generating, by the neural network, the similarity score for each of the submissions collectively comprising the set of similarity scores.

7. The method of claim 1, further comprising receiving additional real signatures associated with the user; and calibrating the model with the additional real signatures.

8. A signature verification system, comprising: one or more processors; and ​ one or more memory devices storing program code configured to be executed by the one or more processors, the program code comprising: a trainer configured to train a model in a plurality of iterations, including, for each iteration of the training, computing a norm of a gradient for each of a first neural network, a second neural network, and a third neural network used for the iteration of the training; based on the computed norm of the gradient, selecting one of the first neural network, the second neural network, and the third neural network to use for the iteration of the training; and updating the selected neural network while leaving other ones of the first neural network, the second neural network, and the third neural network unchanged for the iteration of the training; the model configured to: receive a target signature asserted to be associated with a user; generate a set of similarity scores including a similarity score for each of a plurality of real signatures compared to the target signature; and verify the target signature based on the set of similarity scores.

9. The signature verification system of claim 8, wherein the target signature is classified as real when one or more scores in the set of similarity scores exceed a real classification threshold.

10. The signature verification system of claim 9, wherein the model comprises: a feature extractor configured to: identify a target signature feature for the target signature; and identify a real signature feature for each of the plurality of real signatures.

11. The signature verification system of claim 10, wherein the target signature feature comprises a temporal feature based on timestamps associated with sampling points on a target signature stroke; and wherein the real signature feature comprises a temporal feature based on timestamps associated with sampling points on a real signature stroke.

12. The signature verification system of claim 11, wherein the model further comprises: a transformer-encoder configured to: transform and encode the target signature feature into an encoded target feature; and transform and encode the real signature feature into an encoded real feature for each of the plurality of real signatures.

13. The signature verification system of claim 12, wherein the model further comprises: a classifier configured to: receive the encoded target feature and the encoded real feature for each of the plurality of real signatures; and generate the set of similarity scores including the similarity score for each of the plurality of real signatures represented by the encoded real features compared to the target signature represented by the encoded target feature.

14. The signature verification system of claim 13, wherein the feature extractor comprises the first neural network; wherein the transformer-encoder comprises the second neural network; and wherein the classifier comprises the third neural network. ​ ​ ​ 15. The signature verification system of claim 8, wherein the model is further configured to: receive additional genuine signatures associated with the user; and calibrate the model with the additional genuine signatures.

16. A computer-readable storage medium having program instructions recorded thereon, the program instructions, when executed by processing circuitry, performing a method comprising: for each iteration of a plurality of iterations of training, training a plurality of neural networks using a plurality of genuine signatures associated with a user by: computing an optimality comprising a norm of a gradient for each neural network of the plurality of neural networks; selecting a neural network of the plurality of neural networks having a highest optimality computed; updating the selected neural network while leaving other neural networks of the plurality of neural networks unchanged; providing a model comprising the plurality of trained neural networks with a target signature asserted to be associated with the user and the plurality of genuine signatures; generating, by the model, a set of similarity scores comprising a similarity score for each genuine signature of the plurality of genuine signatures relative to the target signature; and classifying the target signature as genuine or fake based on the set of similarity scores.

17. The computer-readable storage medium of claim 16, wherein the target signature is classified as genuine when one or more scores of the set of similarity scores exceed a genuine classification threshold.

18. The computer-readable storage medium of claim 17, the method further comprising: identifying, by the model, a target signature feature for the target signature; identifying, by the model, a genuine signature feature for each genuine signature of the plurality of genuine signatures in the plurality of genuine signatures; and generating, by the model, the set of similarity scores based on the target signature feature and the genuine signature features.

19. The computer-readable storage medium of claim 18, wherein the target signature feature comprises a temporal feature based on timestamps associated with sampling points on target signature strokes; and wherein the genuine signature feature comprises a temporal feature based on timestamps associated with sampling points on genuine signature strokes.

20. The computer-readable storage medium of claim 19, the method further comprising: encoding, by the model, the target signature feature into an encoded target feature; for each genuine signature of the plurality of genuine signatures, encoding, by the model, the genuine signature feature into an encoded genuine feature; and generating, by the model, the set of similarity scores based on the encoded genuine features and the encoded target feature. ​ ​

Images