Cross-device electronic signature handwriting authenticity identification system, method, device and medium

Abstract

The invention discloses a cross-device electronic signature handwriting authenticity identification system, which unifies a data format through segmented resampling and cepstrum mean variance normalization processing of electronic signature data, and standardizes the data format to ensure the consistency of feature data of different devices on a time axis. A multi-stage one-dimensional depth separable convolutional network is utilized to perform depth feature extraction, and in combination with maximum pooling layer downsampling, the time sequence length and the calculation complexity are reduced. According to the multi-level feature fusion method, features of different depths are superposed, so that richness and robustness of feature representation are enhanced. Feature alignment and similarity measurement are performed through soft alignment, and the authenticity of the signature is judged through the alignment distance, so that the recognition accuracy and robustness of the model under different equipment and cross-time scenes are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a cross-device handwritten electronic information identification method based on deep learning and soft alignment. Background Art

[0002] With the continuous development of digital technology, electronic signature technology has penetrated into all aspects of our daily life, such as online payment, electronic document signing, etc. The core of electronic signature technology lies in how to accurately identify the authenticity of the signature to ensure the validity and security of the signature. In this process, it is usually necessary to compare the signature to be identified with the pre-stored signature of the same user to determine whether the two electronic signature information are written by the same person. The existing electronic signature identification technology mainly extracts the feature points of the signature, and then calculates the similarity between the signature to be identified and the pre-stored signature through a certain algorithm (such as Euclidean distance, Manhattan distance, etc.) to determine the authenticity of the signature to be identified. In addition, there are some deep learning methods that use neural networks to learn the characteristics of the signature to improve the accuracy of identification.

[0003] However, with the widespread application of digital technology, more and more situations require electronic signatures when handling related businesses, logging into related systems, and signing electronic contracts. Different businesses require signature authentication on different electronic devices. The differences in resolution, touch screen sensitivity, and acquisition methods of different devices will affect the quality and consistency of signature data, resulting in a decrease in identification accuracy. The electronic signature recognition methods of the existing technology face many challenges in a cross-device environment. In addition, traditional signature recognition methods do not make sufficient use of the dynamic characteristics of signatures, which wastes the signature information resources obtained by the acquisition device, further limiting its application effect in complex scenarios.

[0004] In the existing electronic signature information authentication system, there are often large differences in signature data between different electronic writing devices (such as mobile phones, signature pads, tablets, etc.). These differences may come from aspects such as device resolution, touch screen sensitivity, and acquisition methods. The difference in collected data makes it very difficult to successfully train the electronic signature recognition model and to ensure the accuracy of electronic signature recognition.

[0005] Collection consistency across devices is a key issue in online signature authentication. If the collection accuracy and sampling rate of different devices are inconsistent, the final authentication effect may be affected. Summary of the invention

[0006] In response to the above-mentioned problems existing in the prior art, the present invention aims to ensure the consistency of data across devices by performing relevant standardized processing on the collected data information of different signing devices, reduce the data differences between different devices through calibration technology or soft alignment technology, and make corresponding compensation for the different collection point sequence data according to the correlation of the signing information, so as to ensure the consistency of the data features signed by the same user across devices and improve the accuracy of authenticity identification of cross-device signing information.

[0007] Based on the first aspect of the present application, a cross-device electronic signature handwriting authenticity authentication system is proposed, including: an acquisition module that acquires time series feature sequences of electronic signature handwriting of different signing devices; a preprocessing module that unifies the time series feature sequence formats of electronic signatures of different devices to obtain normalized feature values ​​of all sampling points; a lightweight network model that extracts local features of the electronic signature handwriting from the normalized feature values ​​and extracts deep features after deep fusion; a soft alignment module that calculates the alignment distance between the corresponding deep features of electronic signature handwriting of different devices; and a recognition module that determines the similarity of electronic signature handwriting collected by different devices based on the alignment distance, thereby determining whether they are written by the same person.

[0008] Further preferably, the preprocessing module unifies the temporal feature sequence format of electronic signatures of different devices through segmented resampling or cepstral mean variance normalization processing; a multi-level deeply separable convolution module is used in the lightweight network model to extract local features of the electronic signature respectively, and a multi-level feature fusion module fuses the outputs of different convolution layers. The maximum pooling layer is used as downsampling to reduce the length of the time series, so as to learn feature representations of different depths to obtain the deep features of the electronic signature.

[0009] Further preferably, the soft alignment module calculates the optimal alignment path of the electronic signature handwriting sequences of different devices through dynamic time warping. Specifically, the soft alignment module calculates the distance between the corresponding elements in the electronic signature handwriting sequences of different devices as the cost function c(xi, yj), and calculates the alignment distance d corresponding to any k-th point and k+1-th point. k d k+1 , calling formula:

[0010]

[0011] Calculate the alignment distance between handwriting sequences, where γ is the smoothing parameter.

[0012] Further preferably, the extracting of deep features after deep fusion includes: the deep separable convolution module calls the formula according to the local features:

[0013] X d [:,n]=X[:,n]*W d [:,n]

[0014] After depth-wise separable convolution, the output feature map X d , all channel output feature maps are concatenated to obtain the fused deep convolution feature map X d , calling the point convolution formula:

[0015] X p =X d *W p

[0016] Calculate feature map X p Get the deep features, where X[:, n] represents the local features of the nth channel input, W d [:,n] represents the nth channel weight, X d [:, n] represents the nth channel output feature map, W p Represents the weight matrix of the point convolution.

[0017] Further preferably, the segmented resampling includes resampling the electronic signature handwriting timing feature sequence into multiple subsequences, calculating the resampled sampling point value for each subsequence, and generating a target sequence by unifying the sampling rate, so that the cross-device electronic signature handwriting timing feature data obtained by different devices remain consistent on the time axis.

[0018] Further preferably, the feature normalization value is calculated based on the cepstral mean variance normalization, and for the i-th feature f in the electronic signature sequence i , calling formula:

[0019]

[0020] Calculate the mean of the i-th feature in the electronic signature sequence and variance Calling formula:

[0021]

[0022] Calculate the normalized value of the i-th feature in the electronic signature sequence Where N is the total number of electronic signature sequence features acquired by the acquisition device, f i (n) It is the i-th characteristic value of the n-th electronic signature sequence characteristic.

[0023] Further preferably, the resampling is divided into a plurality of subsequences including: orig , the target sampling rate r of the sample retention device target , verify the original sampling rate r of the device orig , calling formula:

[0024]

[0025] Determine the target sequence length L after resampling target ; Use the interpolation method to calculate the target sampling point value after resampling, specifically, obtain the target sampling rate r of the sample retention device target , verify the original sampling rate of the device orig , calling formula:

[0026]

[0027] Calculate the nth sampling point information sequence S after resampling target (n), where S orig Represents the original sampling sequence.

[0028] This method is used to resample the electronic signature information collected by different devices, so that the handwritten electronic signature information across devices remains consistent on the timeline, thereby generating a unified complete signature sequence and ensuring the comparability and consistency of the sequence data.

[0029] Based on the second aspect of the present application, a method for identifying the authenticity of electronic signature handwriting across devices is provided, which collects time series feature sequences of electronic signature handwriting from different signing devices; unifies the time series feature sequence formats of electronic signatures from different devices to obtain normalized feature values ​​of all sampling points; extracts local features of the electronic signature handwriting from the normalized feature values ​​and extracts deep features after deep fusion; calculates the alignment distance between the corresponding depth features of the electronic signature handwriting from different devices; determines the similarity of the electronic signature handwriting collected by different devices based on the alignment distance, thereby judging whether they are written by the same person.

[0030] Further optimization is to unify the time series feature sequence format of electronic signatures of different devices through segmented resampling or cepstral mean variance normalization processing; use a multi-level deep separable convolution module to extract local features of the electronic signature respectively, and a multi-level feature fusion module to fuse the outputs of different convolution layers. The maximum pooling layer is used as downsampling to reduce the length of the time series, so as to learn feature representations of different depths to obtain the deep features of the electronic signature.

[0031] Further preferably, the optimal alignment path of the electronic signature handwriting sequences of different devices is calculated by dynamic time warping. Specifically, the soft alignment module calculates the distance between the corresponding elements in the electronic signature handwriting sequences of different devices as the cost function c(x i ,y j ), calculate the alignment distance d corresponding to any k-th point and k+1-th point k d k+1 , calling formula:

[0032]

[0033] Calculate the alignment distance between handwriting sequences, where γ is the smoothing parameter.

[0034] The similarity between signatures is determined based on the distance calculated by the alignment module. The smaller the alignment distance, the closer the handwritten signatures collected by different devices are, and the recognition module determines that they are written by the same person; if the alignment distance is larger, they are determined to be written by different people, thereby effectively improving the accuracy and robustness of cross-device signature recognition.

[0035] Further preferably, the extracting of deep features after deep fusion includes: the deep separable convolution module calls the formula according to the local features:

[0036] X d [:,n]=X[:,n]*W d [:,n]

[0037] After depth-wise separable convolution, the output feature map X d , all channel output feature maps are concatenated to obtain the fused deep convolution feature map X d , calling the point convolution formula:

[0038] X p =X d *W p

[0039] Calculate feature map X p Get the deep features, where X[:, n] represents the local features of the nth channel input, W d [:,n] represents the nth channel weight, X d [:, n] represents the nth channel output feature map, W p Represents the weight matrix of the point convolution.

[0040] Further preferably, the segmented resampling includes resampling the electronic signature handwriting timing feature sequence into multiple subsequences, calculating the resampled sampling point value for each subsequence, and generating a target sequence by unifying the sampling rate, so that the cross-device electronic signature handwriting timing feature data obtained by different devices remain consistent on the time axis.

[0041] Further preferably, the feature normalization value is calculated based on the cepstral mean variance normalization, and for the i-th feature f in the electronic signature sequence i , calling formula:

[0042]

[0043] Calculate the mean of the i-th feature in the electronic signature sequence and variance Calling formula:

[0044]

[0045] Calculate the normalized value of the i-th feature in the electronic signature sequence Where N is the total number of electronic signature sequence features acquired by the acquisition device, f i (n) It is the i-th characteristic value of the n-th electronic signature sequence characteristic.

[0046] Based on the third aspect of the present application, an electronic device includes: a processor; and a memory for storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the cross-device electronic signature handwriting authenticity authentication method according to any one of claims 8-13.

[0047] Based on the fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the cross-device electronic signature handwriting authenticity authentication method according to any one of claims 8-13.

[0048] The present invention proposes a cross-device signature identification method based on deep learning and soft alignment, which standardizes cross-device handwritten electronic signatures, unifies the handwritten electronic information formats of different devices through segmented resampling and cepstrum mean variance normalization, and solves the consistency problem of cross-device signature data: by standardizing the data, the inconsistency problem of signature data between different devices (such as mobile phones, signature pads, tablets, etc.) is effectively solved. It can smooth the differences between devices due to resolution, touch screen sensitivity, and acquisition methods, improve the robustness and reliability of signature identification, ensure the accuracy and robustness of cross-device signature identification, and significantly reduce the risk of misjudgment.

[0049] By designing a lightweight model to extract the deep features of handwritten electronic information, and adding a downsampling module to the model, the standardized handwritten electronic information is downsampled, thereby reducing the length of the time series and reducing the computational complexity of the subsequent soft dynamic time warping (Soft-DTW) alignment process. Further improve performance and efficiency, and provide a solid foundation for subsequent alignment. The alignment distance is calculated for the deep features of different devices. As an important measure of signature similarity, Soft-DTW technology efficiently aligns feature sequences of different signatures, and measures the similarity of signatures through alignment distance, and judges the authenticity of the signature in combination with the threshold set by the signature verification module. This application solves the time complexity and performance issues of signature authentication, greatly improves the accuracy and robustness of signature authentication, and improves the efficiency and performance of signature authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1A schematic diagram of a system for authenticating handwritten electronic information across devices in this exemplary embodiment;

[0051] Figure 2 A schematic diagram of the handwriting recognition model architecture based on soft alignment in this exemplary embodiment;

[0052] Figure 3 A schematic diagram of a lightweight model module for extracting deep handwriting features in this exemplary embodiment;

[0053] Figure 4 A structural block diagram of an electronic device that can be used as a server or client of the present application. DETAILED DESCRIPTION

[0054] Glossary of technical terms:

[0055] DTW (Dynamic Time Warping)

[0056] Soft-DTW (Soft-Dynamic Time Warping)

[0057] CMVN (Cepstral Mean and Variance Normalization) Cepstral Mean Variance Normalization

[0058] The present invention standardizes the characteristics of written information across devices, such as through segmented resampling and cepstral mean variance normalization processing, to unify the electronic signature formats of different devices and ensure the consistency of input data of all different information signing devices; extracts the deep features of electronic signatures, designs a lightweight network model, and adds a downsampling module to further improve performance and efficiency, providing a basis for electronic signature alignment; according to the deep features of different devices, soft dynamic time warping alignment is used to calculate the alignment distance as an important measure of signature similarity; and the final signature authenticity judgment is made based on the alignment distance.

[0059] Figure 1 The figure shows a schematic diagram of a cross-device electronic signature authenticity identification system in an exemplary embodiment of the present application. The acquisition module acquires multi-dimensional feature sequences of electronic information handwriting obtained by different signing devices, unifies the electronic signature formats of different devices through segmented resampling and cepstral mean variance normalization processing, and obtains the normalized value of each feature of all handwriting sampling points; the normalized feature value is input into the lightweight network model, and the multi-layer deep separable convolution module deeply fuses the normalized feature value and extracts the deep features of the electronic signature through the maximum pooling layer; the soft alignment module calculates the alignment distance between the corresponding deep features of different devices, and the recognition module calculates the similarity of the electronic signature based on the alignment distance to determine whether the electronic signatures of different devices are signed by the same person.

[0060] Specifically, the data collection module collects handwritten signature data (which can be sample signatures or verification signature data) from different devices (such as mobile phones, signature pads, and tablets), ensures that the signature samples on each device can reflect natural writing habits, and pre-processes the electronic signature data from different devices, including denoising and feature normalization. For example, through segmented resampling and cepstral mean variance normalization, standard and unified electronic writing information handwriting feature data can be obtained to ensure the consistency of data from various collection devices, improve data quality, provide high-quality feature data for training models, and provide guarantees for dynamic feature alignment of electronic signature verification modules.

[0061] A lightweight network model is designed as a feature extraction module. A downsampling module is added to the model to extract different depth feature representations of electronic signature handwriting. The standardized handwritten electronic signature information is downsampled to reduce the length of the time series and the computational complexity. At the same time, the outputs of different convolutional layers are spliced ​​through multi-level feature fusion to learn feature representations of different depths. In this way, the deep features of handwritten electronic signatures are extracted.

[0062] The cross-device signature verification module uses the soft dynamic time warping (Soft-DTW) module to align the deep features and temporal dynamic features of different handwritten electronic information (including handwritten electronic signatures collected across devices), and calculates the alignment distance between various features.

[0063] The similarity of different electronic signatures is measured based on the alignment distance of all features of two handwritten electronic information calculated by the Soft-DTW module. Based on the set alignment distance threshold, it is determined whether different handwritten electronic signatures are signed by the same person. If the alignment distance is less than the threshold, it is determined that the signatures are signed by the same person; otherwise, it is determined that the signatures are signed by different people.

[0064] The present invention can more accurately obtain deep features that reflect the handwriting characteristics and style of the same signatory by standardizing the signature data features of different acquisition devices. The cross-device signature authentication method based on deep learning and soft alignment significantly improves the accuracy and robustness of signature authentication, and solves the problems of cross-device signature data consistency and electronic signature authentication performance.

[0065] This exemplary embodiment describes in detail the implementation of the present invention through the following specific examples and methods.

[0066] In order to ensure the accuracy and robustness of cross-device signature authentication, the data collection and processing stage must address the challenges brought by specific scene changes such as cross-device. In order to reduce the impact of environmental factors on signature data, data collection from different devices is usually carried out under the same or similar environmental conditions, such as light, writing posture, etc., to ensure that the signature data entering the training and recognition processing is basically unified. For devices that cannot completely unify the environment, pre-processing is used to correct the noise and deviation introduced by environmental changes, thereby ensuring the consistency of training and recognition handwritten signature data.

[0067] First, in terms of signature collection, handwritten electronic signature information is collected from the target user multiple times for different devices (such as mobile phones, signature pads, and tablets), and each signature is completed in a natural state. The collected electronic signature data is used as the authentic signature data, and the authentic signature will be used as the positive sample data for model training. In order to enhance the identification ability of the signature recognition model, samples of others imitating the target user's signature are collected. These samples are imitation signatures, which will be used as negative sample data. The model is trained with positive and negative sample data to improve the accuracy of the network model in distinguishing authentic and imitation signatures.

[0068] Secondly, in terms of device diversification, provide multiple collection devices for the signers, so that the same person can sign the same content through different devices, and try to ensure that the signature covers various device types and obtain signature characteristic data of multiple devices. Collect data samples across devices, for example, through mobile phones, signature pads, tablets and other devices to fully reflect the different resolutions, touch screen sensitivities and writing interaction methods of each device.

[0069] Multiple signature samples are collected for each device to ensure data diversity and coverage. In addition, special consideration is given to differences in device sampling rates and data format compatibility to ensure that signature data from different devices obtain unified sampling frequencies and storage standards.

[0070] In terms of data format, in order to meet the consistency requirements of signature data across devices, the signature data collected by different devices is stored in a unified data storage format. Specifically,

[0071] The handwriting data of cross-device handwritten electronic information can be standardized. Through segmented resampling and cepstrum mean variance normalization, the formats of handwritten electronic information of different devices can be unified to obtain a time-axis consistent sequence of handwritten electronic information time series data. Differences between devices due to resolution, touch screen sensitivity, and acquisition methods can be smoothed. Through data standardization, the inconsistency of signature data between different devices (such as mobile phones, signature pads, tablets, etc.) can be effectively solved. It can improve the robustness and reliability of signature authentication, ensure the accuracy and robustness of cross-device signature authentication, and significantly reduce the risk of misjudgment.

[0072] This exemplary embodiment further illustrates.

[0073] Assume that each handwritten electronic timing feature sequence contains N sampling points, and the i-th sampling point records the following information:

[0074] Sample i ={x i ,y i , p i , t i ,s i ,}

[0075] Where: x i and i Respectively represent the horizontal and vertical coordinates of the i-th sampling point, p i represents the pressure value of the i-th sampling point, t i Indicates the timestamp of the i-th sampling point, s i Indicates the state of the i-th sampling point, where 0 can be set to indicate pen down, 1 to indicate pen moving, and 2 to indicate pen up.

[0076] The electronic handwritten signature sequence can be expressed as:

[0077] Signature={Sample1, Sample2,...,Sample N Implement the following operations:

[0078] In particular, the signature sequence is

[0079] Remove noise and outliers through data preprocessing. For example, you can use mean filtering, median filtering and other methods to remove noise, and remove the sampling point information value of the outlier point by comparing it with the pre-set threshold.

[0080] Standardize the electronic handwritten signature sequence data, such as by resampling to the same sampling rate. First, the signature sequence is divided into multiple subsequences according to the state s of the sampling point, so that each subsequence can be processed segment by segment. Specifically, the segmentation strategy that can be adopted divides the entire electronic handwritten signature sequence into several subsequences according to the characteristics and state changes of the sampling points. Then, resampling operations are performed segment by segment for each subsequence to ensure the uniformity and representativeness of each segment of data, thereby optimizing the subsequent processing steps.

[0081] For example, according to the original sequence length L of the electronic writing information collected from the writing device orig , by reading the target sampling rate r obtained by the sample retention device target , obtain the original sampling rate r by reading the verification device orig , calling formula:

[0082]

[0083] Determine the target sequence length L after resampling target , according to the length L target The retained sample data obtained by different devices are resampled to ensure that the data has the same resolution and data density among different sampling devices.

[0084] Furthermore, in order to better ensure data accuracy and facilitate better alignment between data generated by different devices, the sampling point value of each subsequence after resampling can be calculated using an interpolation method, such as linear interpolation or spline interpolation. Specifically, the target sampling rate r obtained by reading the sample retention device target , obtain the original sampling rate r by reading the verification device orig ,

[0085] Calling formula:

[0086]

[0087] Calculate the nth sampling point information sequence S after resampling target (n), where S orig Represents the original sampling sequence. The sampling point information sequence value here represents the corresponding feature value at each time point, for example, the physical quantities such as pressure, speed, acceleration or angular velocity at each moment in the handwritten signature. By calculating the resampled sampling points, the alignment between the data generated by different devices can be achieved while ensuring the data accuracy.

[0088] Get all resampling points S in each subsequence target (n), further processing includes merging the resampling points of all subsequences into a complete sequence. The subsequences are spliced ​​in the time order of the sampling points, and the endpoints of each subsequence are ensured to be smoothly connected to avoid discontinuity of the sequence data. The merged complete electronic signature sequence is consistent in the time axis to ensure the alignment of the data collected by the retention device and the verification device.

[0089] Based on the cepstrum mean variance normalization (CMVN), the normalized values ​​of different features of signature data collected by different devices are calculated. For example, if the arbitrary feature data at the sampling point of the handwriting data collected by the i-th device is fi (where f i It represents any feature value in the characteristics of the handwriting sampling point data (such as x, y, t, p, v, etc.) obtained by the i-th device, and the calling formula is:

[0090]

[0091] Calculate the mean of the characteristic data of the i-th device and the variance of the characteristic data of the i-th device Thus, the mean μ and variance σ of the characteristic value of any device sampling point are obtained. Where N is the number of samples of all device data, f i (n) is the ith device characteristic value of the nth sample.

[0092] Calling formula: All eigenvalues ​​of the i-th device are normalized.

[0093] Finally, the normalized value of each feature of the handwriting sampling point collected on any device is obtained, such as the normalized value of the x, y coordinates and pressure of device i The normalized characteristic values ​​of the electronic signature sequence data of all collection points of all different collection devices are obtained.

[0094] Each feature data in the signature sequence is normalized based on the cepstrum mean variance to obtain a normalized feature value, and a signature sequence with a consistent time axis is obtained.

[0095] The above method collects and processes the features of cross-device signature handwriting data, and normalizes the data from different collection devices, ensuring the consistency and availability of the model when processing signature data from different devices, providing a high-quality data foundation for the cross-device signature identification model, and effectively responding to the challenges brought by device diversity. The handwriting recognition system accurately judges the authenticity of the signature, ensuring the reliability and accuracy of cross-device signature identification is greatly improved.

[0096] Furthermore, a lightweight convolutional neural network model is constructed to extract the deep features of handwritten electronic signatures. In order to improve efficiency and save computing time, a downsampling module can be added to the convolutional neural network model to reduce the length of the time series, and multi-level feature fusion can be used to construct a multimodal feature representation.

[0097] like Figure 2 Shown is a schematic diagram of the electronic signature handwriting recognition model structure in this exemplary embodiment.

[0098] The collected electronic signature and verification information are normalized and standardized in the above manner, and then input into the deep learning model. Downsampling is added to the deep learning model, and the distance between the two signatures (such as Euclidean distance) is calculated through multi-level feature fusion and soft alignment. The sample signature and verification signature are identified based on the distance to see whether they are written by the same person.

[0099] The amount of computation required by traditional convolutional layers in deep learning models is very large, and the amount of computation is usually determined by the parameters (K and C out ) is determined, the computational complexity of using the traditional convolutional layer and the depth-wise separable convolution is compared as shown in the following formula:

[0100]

[0101] In this formula, K represents the size of the convolution kernel, that is, the spatial dimension of the convolution kernel, and C out It indicates the number of channels of the output feature map. From the above formula, we can see that when K and C out When is larger, the computation of depthwise separable convolution is greatly reduced compared to traditional convolution, thus it can improve the computational efficiency and speed of the model.

[0102] In order to reduce the amount of calculation and effectively extract the local features that reflect the characteristics of the signature handwriting in the signature data, this exemplary embodiment uses a deep separable convolutional layer to fuse and extract handwriting features.

[0103] like Figure 3 Shown is a schematic diagram of the process of extracting deep features by fusing multiple levels of features in an exemplary embodiment of the present application.

[0104] Furthermore, in order to ensure the lightweight of the handwriting recognition model, a one-dimensional depth-separable convolutional layer is constructed as a deep feature extraction model. The convolutional layer can effectively extract local features in the signature data.

[0105] The normalized features of the signature handwriting sampling points (such as x, y, p, t, v...) that have been standardized and normalized preprocessed are sequenced and input into a depth-wise separable convolution module containing multiple channels. The local features of each signature handwriting are extracted through a multi-level depth-wise separable convolution module. Various local features are fused and enter the maximum pooling layer to extract the deep features of the signature handwriting.

[0106] According to local features, such as features The input convolution kernel size is K, and the weight The depth-separable convolution module is:

[0107] X d [:,n]=X[:,n]*W d [:,n]

[0108] Get the output feature map X after depth-separable convolution d , where C in represents the number of input channels, n represents the nth channel, * represents convolution, X d [:, n] represents the nth channel output feature map, W d [:, n] represents the nth channel weight, and X[:, n] represents the nth channel input feature.

[0109] Thus, the deep convolution feature map X of electronic handwriting information is obtained. d , further, the output feature map X is calculated by point convolution p, specifically, calling the point convolution formula:

[0110] X p =X d *W p

[0111] Calculate the fusion feature map X p , where W p Represents the weight matrix of the point convolution.

[0112] In the depthwise separable convolution, the purpose of point convolution is to fuse the features of different channels, help the model learn the correlation between channels, and thus enhance the expressiveness of global features. Point convolution uses a 1x1 convolution kernel to integrate all the information of the input channel into the output channel at each position.

[0113] Using the maximum pooling layer as the downsampling layer can reduce the length of the time series and reduce the calculation complexity of the model. For example, this embodiment uses a three-level deep separable convolution module, and the multi-level feature fusion module superimposes the results of the three convolution modules to construct a multimodal feature representation.

[0114] Furthermore, the SoftDTW (Soft Dynamic Time Warping) module is called to align the deep features of electronic signatures signed at different times and calculate the alignment distance between the two. SoftDTW can handle inconsistent sequence lengths and offsets on the time axis more flexibly, and is suitable for the alignment of dynamic features.

[0115] The alignment path may be a mapping between different sampling points on the time axis. For example, assuming that the alignment path is π, it represents a mapping or path for representing the time matching between the input sequences x and y.

[0116] Specifically, it includes calculating the distance between corresponding elements in the alignment path as the cost function c(x i ,y j ), x and y represent the corresponding elements in the handwriting feature sequences obtained by different acquisition devices, and each sequence represents a handwriting trajectory generated by a specific acquisition device. i ,y j The index corresponding element representing the data point of device i and j in the handwriting trajectory sequence can usually be measured by Euclidean distance or other distance measurement methods that measure the similarity of feature points; calculate the alignment distance d corresponding to the kth point and the k+1th point in the alignment path π k d k+1 , used to measure the difference between adjacent alignment points. Call formula:

[0117]

[0118] Calculate the alignment distance between handwriting information feature sequences collected by different devices.

[0119] Among them, γ is a smoothing parameter used to control the smoothness between paths. A larger γ value will make the alignment path smoother, thereby reducing over-sensitivity to local details.

[0120] During the verification phase, the feature sequence data of the retained signature and the signature to be verified are standardized and normalized, and then input into the deep feature extraction CRNN model. The deep feature extraction model extracts the deep features of the two feature data sequences respectively, and then calls the SoftDTW (soft dynamic time warping) module alignment algorithm to calculate the alignment distance between the features. Next, based on the pre-set threshold τ, the system determines whether the two signatures belong to the same person. If the feature distance is less than the set threshold τ, the signatures are considered to belong to the same person, otherwise they are considered different. Through this process, the system can accurately determine the authenticity of the signature and output the identification result, ensuring the reliability and accuracy of cross-device signature identification.

[0121] This application preprocessing unifies the data format through segmented resampling and cepstral mean variance normalization, and standardizes the signature data to ensure the consistency of feature data of different devices on the time axis. A multi-level one-dimensional deep separable convolutional network is used for deep feature extraction, combined with maximum pooling layer downsampling to reduce the length of the time series and computational complexity. The multi-level feature fusion method superimposes features of different depths to enhance the richness and robustness of feature representation. Feature alignment and similarity measurement are performed through soft alignment, and the authenticity of the signature is judged by the alignment distance, thereby improving the recognition accuracy and robustness of the model in different devices and cross-time scenarios.

[0122] refer to Figure 4 , the structural block diagram of the electronic device 300 that can be used as the server or client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0123] like Figure 4As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0124] A plurality of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, an output unit 307, a storage unit 308, and a communication unit 309. The input unit 306 may be any type of device capable of inputting information to the electronic device 300, and the input unit 306 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 307 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 308 may include, but is not limited to, a disk, an optical disk. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0125] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the reconstruction and decomposition of the muscle movement trajectory redrawn according to the original trajectory of the signature stroke, and the decomposition of its logarithmic velocity curve, etc. can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. In some embodiments, the computing unit 301 can be configured to execute the signature handwriting dynamic acquisition implementation method by any other appropriate means (e.g., by means of firmware).

[0126] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0127] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0131] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0132] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A cross-device electronic signature handwriting authenticity identification system, characterized in that: The collection module collects the temporal feature sequences of electronic signatures of different signing devices; The preprocessing module unifies the time series feature format of electronic signatures of different devices and obtains the normalized feature values ​​of all sampling points; The lightweight network model extracts the local features of the electronic signature handwriting from the normalized feature values ​​and extracts the deep features after deep fusion; The soft alignment module calculates the alignment distance between the corresponding depth features of the electronic signature handwriting of different devices; the recognition module determines the similarity of the electronic signature handwriting collected by different devices based on the alignment distance, thereby determining whether it is written by the same person.

2. The system according to claim 1, characterized in that The preprocessing module unifies the temporal feature sequence format of electronic signatures of different devices through segmented resampling or cepstral mean variance normalization processing; a multi-level deeply separable convolution module is used in the lightweight network model to extract local features of the electronic signature respectively, and a multi-level feature fusion module fuses the outputs of different convolution layers. The maximum pooling layer is used as downsampling to reduce the length of the time series, so as to learn feature representations of different depths to obtain the deep features of the electronic signature.

3. The system according to claim 1 or 2, characterized in that: The soft alignment module calculates the optimal alignment path of the electronic signature handwriting sequences of different devices through dynamic time warping. Specifically, the soft alignment module calculates the distance between the corresponding elements in the electronic signature handwriting sequences of different devices as the cost function c(x i ,y j ), calculate the alignment distance d corresponding to any k-th point and k+1-th point k d k+1 , calling formula: Calculate the alignment distance between handwriting sequences, where γ is the smoothing parameter.

4. The system according to claim 1 or 2, characterized in that: The extraction of deep features after deep fusion includes: the deep separable convolution module calls the formula according to the local features: X d [:,n]=X[:,n]*W d [:,n] After depth-wise separable convolution, the output feature map X d , all channel output feature maps are concatenated to obtain the fused deep convolution feature map X d , calling the point convolution formula: X p =X d *W p Calculate feature map X p Get the deep features, where X[:,n] represents the local features of the nth channel input, W d [:,n] represents the nth channel weight, X d [:,n] represents the nth channel output feature map, W p Represents the weight matrix of the point convolution.

5. The system according to claim 2, characterized in that The segmented resampling includes resampling the electronic signature handwriting timing feature sequence into multiple subsequences, calculating the resampled sampling point value for each subsequence, and generating a target sequence by unifying the sampling rate, so that the cross-device electronic signature handwriting timing feature data obtained by different devices remain consistent on the time axis.

6. The system according to claim 2, characterized in that The feature normalization value is calculated based on the cepstral mean variance normalization. For the i-th feature f in the electronic signature sequence, i , calling formula: Calculate the mean of the i-th feature in the electronic signature sequence and variance Calling formula: Calculate the normalized value of the i-th feature in the electronic signature sequence Where N is the total number of electronic signature sequence features acquired by the acquisition device, f i (n) It is the i-th characteristic value of the n-th electronic signature sequence characteristic.

7. The system according to claim 5, characterized in that The resampling is divided into multiple subsequences including: according to the length L of the original sequence of the collected electronic signature handwriting orig , the target sampling rate r of the sample retention device target , verify the original sampling rate r of the device orig , calling formula: Determine the target sequence length L after resampling target ; Use the interpolation method to calculate the target sampling point value after resampling, specifically, obtain the target sampling rate r of the sample retention device target , verify the original sampling rate of the device orig , calling formula: Calculate the nth sampling point information sequence S after resampling target (n(, where S orig Represents the original sampling sequence.

8. A method for verifying the authenticity of electronic signature handwriting across devices, characterized in that: Collect the temporal feature sequences of electronic signatures from different signing devices; unify the temporal feature sequence formats of electronic signatures from different devices to obtain the normalized feature values ​​of all sampling points; extract the local features of the electronic signatures from the normalized feature values ​​and extract the deep features after deep fusion; Calculate the alignment distance between the corresponding depth features of the electronic signature handwriting of different devices; determine the similarity of the electronic signature handwriting collected by different devices based on the alignment distance, thereby determining whether it is written by the same person.

9. The method according to claim 8, characterized in that The temporal feature sequence formats of electronic signatures of different devices are unified through segmented resampling or cepstral mean variance normalization processing; a multi-level deep separable convolution module is used to extract local features of electronic signatures respectively, and a multi-level feature fusion module fuses the outputs of different convolution layers. The maximum pooling layer is used as downsampling to reduce the length of the time series, so as to learn feature representations of different depths to obtain the deep features of the electronic signature.

10. The method according to claim 8 or 9, characterized in that: The optimal alignment path of the electronic signature handwriting sequences of different devices is calculated by dynamic time warping. Specifically, the soft alignment module calculates the distance between the corresponding elements in the electronic signature handwriting sequences of different devices as the cost function c)x i ,y j ), calculate the alignment distance d corresponding to any k-th point and k+1-th point k d k+1 , calling formula: Calculate the alignment distance between handwriting sequences, where γ is the smoothing parameter.

11. The method according to claim 8 or 9, characterized in that: The extraction of deep features after deep fusion includes: the deep separable convolution module calls the formula according to the local features: X d [:,n]=X[:,n]*W d [:,n] After depth-wise separable convolution, the output feature map X d , all channel output feature maps are concatenated to obtain the fused deep convolution feature map X d , calling the point convolution formula: X p =X d *W p Calculate feature map X p Get the deep features, where X[:,n] represents the local features of the nth channel input, W d [:,n] represents the nth channel weight, X d [:,n] represents the nth channel output feature map, W p Represents the weight matrix of the point convolution.

12. The method according to claim 9, characterized in that The segmented resampling includes resampling the electronic signature handwriting timing feature sequence into multiple subsequences, calculating the resampled sampling point value for each subsequence, and generating a target sequence by unifying the sampling rate, so that the cross-device electronic signature handwriting timing feature data obtained by different devices remain consistent on the time axis.

13. The method according to claim 9, characterized in that The feature normalization value is calculated based on the cepstral mean variance normalization. For the i-th feature f in the electronic signature sequence, i , calling formula: Calculate the mean of the i-th feature in the electronic signature sequence and variance Calling formula: Calculate the normalized value of the i-th feature in the electronic signature sequence Where N is the total number of electronic signature sequence features acquired by the acquisition device, f i (n) It is the i-th characteristic value of the n-th electronic signature sequence characteristic.

14. An electronic device, characterized in that: include: processor; and a memory storing a program, wherein the program includes instructions, which, when executed by the processor, cause the processor to execute the cross-device electronic signature handwriting authenticity authentication method according to any one of claims 8-13.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: in, The computer instructions are used to enable the computer to execute the cross-device electronic signature handwriting authenticity authentication method according to any one of claims 8-13.

Images