Electronic signature verification and anomaly detection system, method and device based on multi-task learning and medium

Abstract

The invention discloses an electronic signature verification and anomaly detection system and method based on multi-task learning, equipment and a medium. The system can perform accurate content identification on the electronic signature sequence, and ensures that the signature information is matched with the user name. And if the content identification result does not accord with the user information, the system analyzes an abnormal track and a dynamic behavior in the signature sequence through a writing abnormity detection module, locates an abnormal signature fragment, and feeds back an abnormal position and a content verification result to the user in real time, thereby realizing accurate abnormity prompt and guidance. According to the system, accurate verification of signature content is achieved through main task output, hierarchical classification and rapid detection of writing abnormity are achieved through auxiliary task output, and the comprehensive verification performance and robustness of electronic signatures are remarkably improved. The system has good expansibility and wide application value, and the security of the electronic signature and the user experience are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an electronic signature verification and anomaly detection technology based on multi-task deep learning. Background Technology

[0002] With the rapid development of information technology and the increasing demand for electronic transactions, electronic signatures are widely used in banking transactions, judicial procedures, and other fields. In e-commerce and transactions, the verification of the signatory's identity relies on an established signature sample database. This database can be created through the signatory's regular signatures or signatures verified through other auxiliary methods. However, traditional electronic signature verification methods often focus on single tasks, such as signature content recognition or identity verification, lacking a comprehensive detection mechanism for handwriting anomalies. This limitation results in a lack of effective anomaly alerts when the signature content does not match the username, easily leading to poor user experience and high risk of disputes. Furthermore, existing electronic signature verification methods typically employ a single model, which also has limitations in handling complex handwriting anomalies. For example, issues such as tilted signing angles, missing strokes, and missing or extra characters are difficult to accurately capture and identify. These anomalies not only affect the validity of the signature but may also lead to subsequent identity verification failures, thus affecting the smooth progress of the entire transaction process.

[0003] To address these issues, a multi-task system is urgently needed that can simultaneously perform signature content and username consistency verification and write anomaly detection and classification. This system should be able to effectively extract the spatiotemporal and semantic features of the signature sequence and, through joint training with a multi-task loss function, achieve efficient collaboration between the main and auxiliary tasks, thereby improving verification accuracy, enhancing user experience, and reducing the risk of disputes.

[0004] For example, Chinese patent application CN115937873A discloses an online handwriting verification method based on recognizable single characters. This method collects individual handwritten characters or divisible characters from preset content, determines whether the signed content matches the preset characters based on character recognition, and trains a single-character handwriting classification or representation network model with characters identical to the preset content and carrying identification identifiers. This aggregates samples of the same person and the same characters in the vector space, while distancing other samples. The method extracts character representations and compares the similarity of the representation vectors of the sampled characters and the verification characters to determine whether the characters were written by the same person. However, this method lacks the ability to perform content verification and semantic understanding of the character data.

[0005] Chinese patent application CN112651410A discloses a model training and identification method for authentication. This method acquires training sample images containing image regions corresponding to at least one authentication point of the item to be authenticated. It then processes these training sample images using an object detection model to obtain object detection results. These results are then used to train a classification model, outputting the classification result of the part of the item to be authenticated corresponding to the authentication point, as well as the authenticity verification result. However, this method still requires optimization in its machine learning model training approach, lacking further improvement in the model's authentication capability and efficiency.

[0006] Existing technologies typically identify whether the written characters are genuine or whether the written content matches the sample characters. They lack multiple checks on the written content and errors, and lack effective methods to simultaneously optimize and coordinate the relationships between different tasks. Their implementation of multi-task learning is also insufficient. This makes it difficult for current signature recognition models to achieve optimal performance and efficiency when handling multiple tasks such as signature content recognition and error detection. They also fail to fully utilize the potential correlations between tasks, thus affecting the overall system performance and reliability.

[0007] Existing technologies in electronic signature sequence content verification systems lack optimization in data processing, particularly in character data content verification and semantic understanding. This limits the accuracy and efficiency of electronic signature recognition models when handling complex signature content, making it difficult to fully mine and utilize the deep semantic information of the signature sequence. Existing electronic signature verification methods also have limitations in handling complex handwriting anomalies and their ability to detect anomalies caused by changes in writing style and environment. For example, issues such as tilt angles, missing strokes, and missing characters are difficult to accurately capture and identify with a single model. These anomalies not only affect the validity of the signature but also easily lead to subsequent identity verification failures, thus affecting the smooth operation of e-commerce transactions. Therefore, traditional electronic signature verification methods often focus on single tasks, such as signature content recognition or identity verification, lacking a comprehensive detection mechanism for handwriting anomalies. This limitation means that when the same signer uses connected strokes or abbreviations, resulting in inconsistencies between the signature content and the username, there is a lack of a comprehensive verification mechanism for the electronic signature sequence and content, and a lack of an effective verification mechanism for handwriting anomalies (such as missing strokes, tilting, and missing characters). This inability to effectively identify and alert to handwriting anomalies leads to poor user experience and a high risk of disputes. Especially when dealing with multi-character or complex character sets, existing methods struggle to accurately identify and process them, impacting the overall accuracy and efficiency of the verification process. Summary of the Invention

[0008] Existing electronic signature verification systems mostly focus on verifying single pieces of information and lack effective comprehensive detection and verification mechanisms. This makes it difficult to detect signature anomalies in a timely manner, increases the difficulty of accurately obtaining and establishing user signature data samples, and results in a poor user experience, increasing the risk of signature disputes.

[0009] To address the aforementioned problems in existing technologies, this invention provides an electronic signature sequence content verification and writing anomaly detection system based on multi-task learning. The system includes a shared feature extraction module, a multi-task joint training module, a content verification and anomaly detection collaboration module, and an anomaly prompting and user feedback module. The shared feature extraction module extracts multi-dimensional temporal features of the electronic signature, capturing long-distance dependencies and spatiotemporal relationships in the signature sequence to generate a high-dimensional shared feature representation. The multi-task joint training module determines the joint loss function of the main task and auxiliary tasks, dynamically adjusts task weights, and jointly optimizes signature content verification and writing anomaly detection. In the content verification and anomaly detection collaboration module, the main task output head extracts electronic signature spatial features based on the shared feature representation and identifies the signature content. The auxiliary task output head performs hierarchical classification of writing anomalies and detects writing anomalies in real time based on the shared feature representation. The anomaly prompting and user feedback module outputs the content verification and writing anomaly detection results.

[0010] Further optimization involves extracting spatial features from the shared feature representation from the main task output head, inputting them to the connection-series classification CTC unit for decoding to complete the sequence recognition of the signature content. The CTC unit decoder performs dynamic programming search on the output probability distribution of the signature sequence at each time step, selects the character sequence with the highest overall conditional probability, and maps the signature temporal features to the text content sequence.

[0011] Further optimization involves using shared features to optimize the auxiliary task output head. The input is a multi-layer convolutional neural network. Multi-level feature extraction and downsampling are performed on shared features. Each network layer extracts hidden features related to anomaly categories from the signature sequence. These hidden features are then pooled to output salient features reflecting various anomaly patterns. Fully connected layers map these salient features to a discriminative space, achieving accurate classification of anomaly types. Specifically, the multi-level feature extraction and downsampling of shared features includes convolutional layers capturing changes in handwriting through local receptive fields, pooling layers spatially compressing features, and identifying abnormal shapes and structures in character handwriting.

[0012] Further optimization is performed according to the formula: Calculate the loss function of the main task output header, where, For the target signature text sequence, The sequence probability predicted by the model; according to the formula: Calculate the probability distribution of multiple anomaly categories. According to the formula: Calculate the loss of the auxiliary task, where C is the total number of anomaly categories.

[0013] Further optimization is achieved through a weighted summation method, according to the formula: Calculate the loss function of the multi-task joint training module, where, The weight parameter represents the priority of the main task. The initial value is set to ensure that the main task objective takes precedence.

[0014] According to a second aspect of the present invention, a method for verifying the content of electronic signature sequences and detecting writing anomalies based on multi-task learning is proposed, comprising: extracting multi-dimensional temporal features of electronic signatures; capturing long-distance dependencies and spatiotemporal relationships in the signature sequence to generate a high-dimensional shared feature representation; determining the joint loss function of the main task and the auxiliary task; achieving collaborative optimization of the two tasks; automatically adjusting task weights according to the loss changes during training to ensure the priority of the main task; and extracting electronic signature spatial features from the outputs of the two independent tasks based on the shared feature representation, identifying the signature content, classifying writing anomalies hierarchically, and detecting writing anomalies in real time.

[0015] Further optimization involves extracting spatial features from the shared feature representation from the main task output head, inputting them to the connection-series classification CTC unit for decoding to complete the sequence recognition of the signature content. The CTC unit decoder performs dynamic programming search on the output probability distribution of the signature sequence at each time step, selects the character sequence with the highest overall conditional probability, and maps the signature temporal features to the text content sequence.

[0016] Further optimization involves using shared features to optimize the auxiliary task output head. The input consists of multiple convolutional neural network layers. Multi-level feature extraction and downsampling are performed on shared features. Each network layer extracts hidden features related to anomaly categories from the signature sequence. These hidden features are then pooled to output salient features reflecting various anomaly patterns. Fully connected layers map these salient features to a discriminative space, achieving accurate classification of anomaly types. Specifically, the multi-level feature extraction and downsampling of shared features includes convolutional layers capturing changes in handwriting through local receptive fields, pooling layers performing spatial compression of features, and identifying abnormal shapes and structures in character handwriting. According to a third aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the electronic signature sequence content verification and writing anomaly detection method based on multi-task learning described above.

[0017] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the electronic signature sequence content verification and writing anomaly detection method based on multi-task learning according to any one of the preceding claims.

[0018] This invention constructs a unified electronic signature sequence feature extraction network and two independent output nodes to achieve simultaneous processing of signature content verification and writing anomaly identification. It effectively solves the problem that traditional electronic signature content identification methods focus too much on a single task. By combining a shared feature extraction module with two independent task output heads, it achieves accurate verification of the consistency between the electronic signature content and the username, as well as real-time detection and prompting of writing anomalies. This significantly improves the comprehensiveness and accuracy of verification, thereby enhancing the security of electronic signatures and the user experience.

[0019] Because the amount of time-series data sampled for electronic signatures is limited, yet contains a large amount of information, improving recognition accuracy requires significantly increasing the granularity of data collection, resulting in a substantial increase in computational load. Furthermore, multi-task processing makes it difficult to effectively allocate the workload and responsibilities for recognition. This invention addresses the shortcomings of existing multi-task learning technologies, such as difficulty in effectively coordinating task priorities in complex and segmented scenarios like electronic signature recognition and sample collection, leading to poor recognition accuracy and user experience. This invention employs a sequence recognition module to achieve high-precision recognition of signature content, supplemented by a classification network for hierarchical detection and real-time capture of writing anomalies (such as tilt angles, insufficient strokes, and missing characters). The optimized loss function dynamically adjusts the weights of primary and secondary tasks, significantly enhancing the accuracy and reliability of signature verification. A constant prompting mechanism provides immediate feedback to users on signature issues based on the writing anomaly classification results, effectively improving user interaction and reducing the risk of signature disputes. Simultaneously, based on a unified shared feature extraction network and standardized model structure, the system possesses excellent scalability and maintainability, flexibly adapting to new anomaly categories and constantly changing business needs. Attached Figure Description

[0020] Figure 1 This exemplary embodiment illustrates the electronic signature sequence content verification and writing anomaly detection based on multi-task learning. Figure 2 A schematic diagram of the collaborative module for multi-task learning content verification and anomaly detection in this exemplary embodiment; Figure 3 This can serve as a block diagram of the electronic device structure for the server or client in this application. Detailed Implementation

[0021] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0022] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0023] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0024] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0026] This invention aims to overcome the shortcomings of existing technologies by combining a shared feature extraction module with two independent task output heads to achieve accurate verification of the consistency between the electronic signature content and the username, as well as real-time detection and prompting of writing anomalies, thereby improving the security of electronic signatures and user experience.

[0027] This invention collects multi-dimensional time-series signature data and anomaly annotations to ensure the integrity of training data. Based on a unified feature extraction network, it jointly optimizes signature content verification and writing anomaly detection, dynamically adjusting task weights while prioritizing the main task. The trained model accurately identifies signature content and detects writing anomalies in real time, achieving efficient coordination between the two tasks. Based on the detection results, it provides intuitive feedback on the location and type of abnormal signatures, helping users quickly correct signature problems. The overall system structure is concise and functionally complete, effectively improving the security of electronic signatures and the user experience.

[0028] An electronic signature verification and anomaly detection system based on multi-task learning includes a shared feature extraction module, a multi-task joint training module, a content verification and anomaly detection collaboration module, and an anomaly prompting and user feedback module. The shared feature extraction module processes the multi-dimensional time-series raw data of the electronic signature (including pen tip x / y...). The system performs deep encoding of coordinates, writing pressure, pen tip angle, and sampling timestamps. By capturing the long-distance temporal dependencies and spatiotemporal relationships of the signature sequence, it generates a high-dimensional shared feature representation that simultaneously supports the main task (signature content verification) and the auxiliary task (writing anomaly detection). The multi-task joint training module defines dedicated loss functions for the main task and the auxiliary task, and achieves collaborative optimization of the two tasks by dynamically adjusting the task weights (adaptively optimizing based on the convergence speed of task loss). While ensuring the priority of the main task, it enhances the adaptability of the shared features to the two tasks. In the content verification and anomaly detection collaborative module, the main task output head extracts spatial core features (including coordinate trajectory curvature, key feature point spacing, and spatial distribution of writing state) from the high-dimensional shared feature representation and inputs them into the Connectionist Temporal Classification (CTC) unit to complete the signature content sequence recognition. The auxiliary task output head detects writing anomalies in real time and completes accurate anomaly type classification based on the high-dimensional shared feature representation through a hierarchical feature extraction and classification mechanism. The anomaly prompt and user feedback module synchronously outputs the signature content verification result (correct / incorrect) and the writing anomaly detection result (anomaly type / no anomaly), and generates interpretable feedback information for anomaly situations.

[0029] The main task output head has a built-in spatial feature extraction branch. Through local feature enhancement and dimensionality filtering, it separates the signature spatial structure and trajectory association features from the high-dimensional shared feature representation. The CTC unit uses the Viterbi dynamic programming algorithm to perform a global optimal search on the character probability distribution at each time step of the signature time sequence, selects the character sequence with the highest overall conditional probability, solves the problem of inconsistent length between the "high-dimensional shared feature time sequence" and the "signature text content sequence", and realizes accurate mapping from time-space fusion features to text content sequence.

[0030] The auxiliary task output head consists of multiple layers of alternating convolutional layers, pooling layers, and subsequent fully connected layers. The multi-layer convolutional neural network gradually strengthens anomaly correlation information through multi-level feature extraction and downsampling: the convolutional layers capture subtle writing deviations (such as abrupt angle changes and stroke breaks) through local receptive fields, generating local / global hidden features strongly correlated with the anomaly category; the pooling layers spatially compress the hidden features while preserving key information, enhancing the model's robustness to anomalies such as shape distortion and structural defects; the final output salient features centrally reflect the core identifiers of anomaly patterns such as angle anomalies, missing strokes, and missing characters; the fully connected layers map these salient features to the anomaly category discrimination space built into the auxiliary task (the dimension corresponds one-to-one with the number of anomaly categories), achieving hierarchical and accurate classification of multiple anomalies. Specifically, the following examples further illustrate the functional implementation of each module of this system.

[0031] The shared feature extraction module collects multi-dimensional temporal signature data (including position, pressure, speed, angle, etc.) and anomaly labels (such as "missing strokes" and "tilted angle") to provide complete data for subsequent training. After preprocessing, the extracted data will be used as the input source for the shared feature extraction network.

[0032] The multi-task joint training module is the core carrier for training model parameters. It includes a shared feature extraction network, a main task output head (dedicated to content verification), and an auxiliary task output head (dedicated to anomaly detection). This module performs deep feature modeling on preprocessed handwriting data based on the shared feature extraction network. Through a dynamic weighted joint loss function (ensuring priority for the main content verification task), it simultaneously optimizes the parameters of the three components. This allows the shared feature extraction network to generate high-dimensional features suitable for both tasks, while also enabling the main output head (such as the CTC decoding unit) to accurately identify signature text and the auxiliary output head (such as a multi-layer CNN) to accurately classify handwriting anomalies.

[0033] The content verification and anomaly detection collaborative module is the "core carrier of real-time detection." It calls upon the shared feature extraction network optimized by the multi-task joint training module and the main / auxiliary task output headers. Without repeated training, it directly loads the pre-trained component parameters and performs parallel inference on the real-time input signature data, performing "shared feature extraction → main task content verification → auxiliary task anomaly detection," achieving efficient coordination between the two tasks.

[0034] The anomaly alert and user feedback module: Based on the content verification results (such as text matching with identity) and anomaly detection results (such as missing strokes of the second character) output by the collaboration module, it provides intuitive feedback on the location and type of the anomaly, helping users quickly correct signature problems.

[0035] Figure 1 illustrates the electronic signature sequence content verification and writing anomaly detection based on multi-task learning in an exemplary embodiment of this application, specifically including: The system acquires the original electronic signature data and username, preprocesses the data, and extracts features. The preprocessed data is then input into a shared feature extraction network. The shared feature space, optimized by the multi-task learning and training module, generates the high-dimensional features that support both tasks. Signature recognition is performed through the main task output branch (the main task output head), which, based on features from the shared feature space, uses CTC temporal decoding to recognize the signature text (e.g., mapping temporal features to "Zhang San"). Anomaly detection is performed through the auxiliary task output branch (the auxiliary task output head), which, based on features from the same shared feature space, extracts anomalous features using a multi-layer CNN to classify and locate anomalies such as "missing strokes" and "tilted angles." The signature recognition and anomaly detection results are then jointly trained across multiple tasks. By comparing the results from both output heads with the labeled values, a joint loss function is calculated, and the parameters of the shared feature extraction network and both output heads are updated in reverse, completing iterative model optimization.

[0036] First, the input electronic signature sequence is preprocessed. This includes standardizing the signature sequence and normalizing it to a fixed-size two-dimensional space. The electronic signature sequence features include spatiotemporal information such as starting point coordinates, length, width, and angle, while also capturing dynamic features of the handwriting, such as pressure and speed. The temporal features at this stage are represented as a set of vectors: in, For the number of strokes Two-dimensional coordinates of a point For width, For angle, This is the pressure value. For speed.

[0037] To enhance the system's ability to identify writing errors, a writing error data annotation mechanism is further introduced. This mechanism annotates writing error data with reference to standard characters. Error annotations can be obtained through the following two methods: 1) Data Collection and Annotation – The collected electronic signatures can be checked manually or by computer, and annotated according to the standard sample font and shape. The types of signature anomalies are marked, including: missing strokes, abnormal tilt angle, missing multiple characters, and abnormal position. Annotating the signature anomalies can effectively guide model training.

[0038] 2) Data Augmentation Synthesis – Based on multiple signatures of the same character signed by different people stored in the signature sample library, samples with synthetic anomalies are generated to simulate various writing anomalies such as missing strokes and different stroke tilt angles. This generates an abnormal training data sample set, expanding the dataset diversity and improving the model's generalization ability to writing anomalies. Simultaneously, the anomaly types of the data obtained from data augmentation synthesis are also labeled. By integrating labeling and data synthesis, a rich and high-quality training and testing dataset is formed, effectively improving the accuracy and robustness of anomaly detection and providing a solid data foundation for subsequent multi-task learning and training.

[0039] The multi-task joint training module designs and trains a joint task model based on the collected and labeled, data-augmented multidimensional time-series electronic signature data. First, a shared feature extraction network is used to perform deep modeling of long-distance dependencies and spatiotemporal dynamic features in the signature sequence, generating a high-dimensional shared feature representation.

[0040] Two independent output heads are constructed to process the shared features respectively. The main task output head performs signature content verification based on the shared features, while the auxiliary task output head performs hierarchical classification and real-time detection of writing anomalies based on the shared features.

[0041] Specifically, the encoder in the shared feature extraction network performs deep fusion of signature temporal features to generate a high-dimensional shared feature representation. "Shared features" refer to a unified feature vector or feature matrix obtained after encoder processing, capable of representing the essential information of the signature sequence, and usable by both the primary task (content verification) and the secondary task (writing anomaly detection). This shared feature approach addresses the needs of both tasks, avoids redundant computation, and improves model training and inference efficiency.

[0042] Obtaining the multidimensional time sequence of electronic signatures Each time step contains multi-dimensional features (such as handwriting position, pressure, speed, angle, etc.) input to the encoder, such as a shared feature extraction network built based on a bidirectional LSTM, GRU, or Transformer encoder. Perform time series modeling. It includes a 6-layer bidirectional Transformer encoder, with each layer containing 8 self-attention heads and 2 fully connected layers: l Self-attention focus: Captures the dependency relationship between any two time points in the feature matrix X, solving the problem that features at a single time point cannot reflect the "stroke continuity"; l Fully connected network: Maps the "dependent features" of self-attention output to a higher dimension, enhancing the discriminative ability of the features.

[0043] Extracting high-dimensional shared feature sequences from electronic signatures in, This indicates that the high-dimensional shared feature sequence extracted by the encoder can simultaneously support the main task (content verification) and the auxiliary task (anomaly detection). It is a set of learnable parameters for the shared feature extraction network, including: the weight matrix of the self-attention head, and the weights and biases of the fully connected network, totaling approximately 12 million parameters. During training, it is continuously updated through backpropagation of the joint loss function. The encoder generates It can more accurately capture the "temporal patterns of normal signatures" and the "characteristic mutations of abnormal signatures".

[0044] The core function of the main task output head is to extract high-dimensional shared feature sequences from the encoder. Spatial features of the signature are extracted, then fused with temporal information and input into the Connectionist Temporal Classification (CTC) unit to decode and complete the recognition of the signature text content sequence. It is a high-dimensional feature set obtained by the encoder after performing deep encoding on the original electronic signature data (such as pen tip x / y coordinates, writing pressure, pen tip tilt angle, sampling timestamp, etc.). Each element All contain temporal correlation information and basic feature representations of the signature, and serve as core shared resources for the multi-task system. They also provide feature support for the main task of signature sequence content verification and the auxiliary task of writing anomaly detection. The main task output header, through a built-in spatial feature extraction branch, extracts features from each element. Spatial attribute enhancement, dimensional filtering, and geometric feature decoding are performed. The extracted spatial features include: ① Coordinate trajectory features (coordinate difference between adjacent sampling points, trajectory curvature, inflection point position and density, etc.); ② Spatial state features (spatial distribution gradient of writing pressure, spatial variation pattern of pen tip angle, etc.); ③ Spatial structure features (spacing of key signature feature points, relative layout between characters, spatial proportion of the overall writing area, etc.), ultimately forming a target feature sequence that integrates spatial and temporal information. — That is, the core input of implicit spatial features, through Establishing a probabilistic correlation between spatial features and target sequences directly reflects the supporting role of spatial features in sequence recognition. The sequence to be recognized here specifically refers to the text content sequence corresponding to the target signature (such as the name of the signer, discrete character combinations of specific signed text, and other core content that needs to be verified).

[0045] The loss function of the CTC unit is defined as: in Sign the target text sequence (real label). Predicting sequences for the model Relative to the input feature sequence The conditional probability. Specifically: This represents the CTC loss function, whose core function is to quantify the difference between the model's prediction results and the actual signature text sequence, providing a backpropagable loss metric for model training. The model to which this function belongs is the CTC classification model, which is the core computation module of the signature sequence content verification main task in this multi-task system.

[0046] The auxiliary task output head uses a multi-layer convolutional neural network to perform multi-level feature extraction and downsampling on shared features, gradually enhancing the local and global information related to anomalies in handwriting sequence feature points. The core process of anomaly feature enhancement and classification is completed through a multi-layer convolutional neural network (CNN, including alternating convolutional and pooling layers): Feature processing: Convolutional layers capture subtle writing anomaly signals (such as angle deviations and missing strokes) through local receptive fields, while pooling layers compress the feature space to enhance the robustness of structural anomalies. The processing of shared features focuses on "purification of anomaly-related information"—filtering redundant content and gradually fusing local (subtle deviations) and global (stroke / structural anomalies) information. The generated hidden features are "anomaly pattern association features," and the final output salient features are the core identification information of various anomalies (angle, stroke, structural anomalies, etc.).

[0047] Based on shared features Input to a multi-layer convolutional neural network: The hidden features of the network layers, after multiple convolution and pooling operations, are represented by the output feature representation, which reflects the salient features of various abnormal patterns.

[0048] Accurate Classification and Module Relationship: Significant features are mapped through fully connected layers to the "anomaly category discrimination space" built into the auxiliary task output head (the dimension is consistent with the number of anomaly categories). The conditional probability distribution of each anomaly category is calculated by operating on the mapped feature vectors using the Softmax function (the operation objects are significant features and fully connected layer weights / biases, and the parameters are obtained through training optimization). The core role of the probability distribution is to select the category with the highest probability as the final anomaly type, while providing input for loss calculation; it is subsequently linked with the main task results to complete double verification. Module Hierarchy: Convolutional layers, pooling layers, fully connected layers, and the discrimination space together constitute the auxiliary task output head, presenting a hierarchical relationship of "feature extraction → high-dimensional mapping → category discrimination".

[0049] The loss of the auxiliary task can be calculated using cross-entropy: Where C is the total number of anomaly categories. For example, C=4 corresponds to the four categories of "angle anomaly, missing strokes, missing multiple characters, and no anomaly". The specific value is determined by the actual anomaly type classification requirements. The salient features extracted by multi-layer CNN for the auxiliary task; The one-hot true label for the i-th type of anomaly (obtained from manually labeled training data); Predict the conditional probability (Softmax output) for the model that the signature belongs to the i-th type of anomaly. The multi-task joint training module first defines corresponding loss functions for the main task and auxiliary task, sets priority weights for the main task, and ensures optimal optimization of the main task objective. For example, it uses a weighted summation of the main task loss L and the auxiliary task loss. Combining these two losses yields the joint loss function: The primary task priority weight (initially manually set, can be adaptively adjusted) serves as the global optimization target to balance the primary (content verification) and secondary (anomaly detection) tasks, and in turn updates the parameters of the entire system to achieve collaborative optimization.

[0050] During training, the multi-task learning module monitors the changes in the loss values ​​of the primary and secondary tasks in real time, calculating the trends of current and historical loss (such as gradient direction or loss descent rate) using methods like sliding window or exponential moving average. Based on this information, a dynamic weighting strategy is employed to adjust the weight parameters. Specifically, it includes: If the main task's loss decreases slowly or fluctuates, the system will increase the weighting parameters. The value is increased to strengthen the weight of the main task, prompting the model to focus more on optimizing the main task.

[0051] If the loss of the auxiliary task decreases rapidly and the performance of the primary task remains stable, the weighting parameters should be appropriately reduced. This enhances the contribution of auxiliary tasks and improves the overall generalization ability.

[0052] The adjustment process follows the principle of smoothness to avoid excessive weight fluctuations that could affect training stability.

[0053] Through this dynamic weight adjustment mechanism based on loss change feedback, the joint loss function achieves collaborative optimization of the main and auxiliary tasks during training, while ensuring that the main task always maintains priority, thereby improving the overall performance and training efficiency of the training model.

[0054] refer to Figure 2In this exemplary embodiment, a schematic diagram of a collaborative module for content verification and anomaly detection based on multi-task learning is provided. The multi-task learning training module uses electronic signature handwriting sequence data that has been collected, labeled, and data augmented to integrate and train the neural network model to construct the collaborative module for content verification and anomaly detection, and performs content verification and anomaly detection on the electronic signature sequence input.

[0055] The collaborative module for electronic signature content verification and anomaly detection first extracts high-dimensional shared features from the input signature sequence through a shared feature extraction network, capturing the long-distance dependencies and spatiotemporal dynamic information of the electronic signature. The main task output head identifies and verifies the signature content based on the shared features, comparing the identified content with the user's identity binding information to determine if the signature content is abnormal. When the signature content verification result is abnormal, the auxiliary task output head uses a multi-layer convolutional network to extract abnormal features and perform multi-level classification on the shared features, detecting and locating writing anomalies in the signature, such as missing strokes, tilted angles, and missing characters. The content verification and anomaly detection tasks work in parallel and collaboratively, sharing the same high-dimensional feature representation, improving the overall accuracy of signature recognition and the precision of anomaly detection.

[0056] The electronic signature content verification and anomaly detection collaborative module outputs signature content verification results and anomaly detection prompts, while the anomaly prompt and user feedback module outputs content verification and writing anomaly detection results for subsequent system security decisions and user interaction. It does not automatically correct signature content or anomalies; instead, it relies on anomaly prompts to assist users or the system in risk assessment or signature rewriting, thereby ensuring the correctness of the signature content and the legality of the writing action.

[0057] The core advantage of this invention lies in its effective combination of content recognition and handwriting anomaly detection through multi-task shared feature modeling. This not only ensures the accuracy of signature content recognition but also enables real-time and precise detection of various abnormal behaviors, enhancing the anti-counterfeiting security and user experience of the electronic signature system. This system fully utilizes the spatiotemporal modeling capabilities of the shared feature extraction module and the specialized recognition capabilities of the multi-task output head to achieve accurate verification of electronic signature content and effective detection of handwriting anomalies, thereby improving system security and user experience.

[0058] This invention relates to a method for verifying electronic signature sequence content and detecting writing anomalies based on multi-task learning. It extracts multi-dimensional temporal features of electronic signatures, captures long-distance dependencies and spatiotemporal relationships in the signature sequence to generate a high-dimensional shared feature representation, determines the joint loss function of the main task and auxiliary tasks, and achieves collaborative optimization of the two tasks. The method automatically adjusts task weights based on loss changes during training to ensure the priority of the main task. Two independent task output units extract spatial features of the electronic signature based on the shared feature representation, identify the signature content, perform hierarchical classification of writing anomalies, and detect writing anomalies in real time.

[0059] Through multi-task collaborative learning, accurate verification of electronic signature sequence content and effective detection of writing anomalies are achieved, significantly improving the overall performance and user experience of signature verification, and possessing good scalability and practical value.

[0060] refer to Figure 3 The present invention describes a structural block diagram of an electronic device 300 that can serve as a server or client of this application, which is an example of a hardware device that can be applied to various aspects of this application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or claimed herein.

[0061] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0062] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, output unit 307, storage unit 308, and communication unit 309. Input unit 306 can be any type of device capable of inputting information to electronic device 300. Input unit 306 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 307 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 308 may include, but is not limited to, disk and optical disk. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0063] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the reconstruction and decomposition of the muscle movement trajectory of the signature stroke based on the original trajectory, and the decomposition of its logarithmic velocity curve, can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. In some embodiments, the computing unit 301 can be configured by any other suitable means (e.g., by means of firmware) to perform a signature handwriting dynamic acquisition implementation method.

[0064] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0065] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0066] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0067] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0068] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0069] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0070] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.

Claims

1. A multi-task learning-based electronic signature sequence content verification and writing anomaly detection system, characterized in that, Comprise: The shared feature extraction module, multi-task joint training module, content verification and anomaly detection collaborative module, anomaly prompt and user feedback module; the shared feature extraction module encodes the multi-dimensional time sequence original data of electronic signature, captures the long-distance time sequence dependence and space-time correlation of the signature sequence, and generates a high-dimensional shared feature representation that supports the main task and the auxiliary task; the multi-task joint training module determines the exclusive loss function for the main task and the auxiliary task, and realizes the collaborative optimization of the two tasks by dynamically adjusting the task weight; in the content verification and anomaly detection collaborative module, the main task output head extracts the spatial core features of the electronic signature from the high-dimensional shared feature representation, and inputs the connection time sequence classification unit to complete the signature content sequence identification; The auxiliary task output head detects the writing anomaly in real time and completes the anomaly type division based on the high-dimensional shared feature representation through the hierarchical feature extraction and classification mechanism; the anomaly prompt and user feedback module synchronously outputs the signature content verification result and the writing anomaly detection result, and generates explainable feedback information for the abnormal situation.

2. The system of claim 1, wherein, The main task output head has a built-in spatial feature extraction branch, which separates the signature spatial structure and trajectory correlation features from the high-dimensional shared feature representation through local feature enhancement and dimension selection; the CTC unit adopts the Viterbi dynamic programming algorithm to perform global optimal search on the character probability distribution of each time of the signature time sequence, selects the character sequence with the highest overall conditional probability, and realizes the accurate mapping of the time-space fusion feature to the text content sequence.

3. The system of claim 1, wherein, The auxiliary task output head includes a multi-layer convolutional neural network composed of multiple layers of alternating stacked convolutional layers, pooling layers and fully connected layers, which gradually strengthens the abnormal correlation information through multi-level feature extraction and down-sampling; the convolutional layer captures the subtle deviation in writing through local receptive field, and generates local / global hidden features strongly related to the abnormal category; the pooling layer performs spatial compression and key information retention on the hidden features to output significant features; the fully connected layer maps the significant features to the abnormal category discrimination space built-in the auxiliary task, and realizes hierarchical accurate classification of multiple abnormal categories.

4. The system according to one of claims 1-3, characterized in that, The multi-task joint training module determines the exclusive loss function for the main task and the auxiliary task, including calculating the loss of the auxiliary task according to the formula: The main task of computing the output head loss function, wherein, For the target signature text sequence, For the sequence probability predicted by the model; according to the formula: Computing probability distributions for multiple anomaly classes , according to the formula: The loss of the auxiliary task is calculated according to the formula:

5. The system of claim 4, wherein, Through weighted summation, according to the formula: The loss function of the multi-task joint training module is calculated, wherein, The weight parameter representing the priority of the main task is initially set to ensure that the target of the main task is given priority.

6. A method for electronic signature sequence content verification and writing anomaly detection based on multi-task learning, characterized in that, Including, extracting multi-dimensional time sequence features of electronic signature, capturing long-distance dependence and space-time relationship in signature sequence to generate high-dimensional shared feature representation, determining joint loss function of main task and auxiliary task, realizing collaborative optimization of two tasks, automatically adjusting task weight according to loss change in training process to ensure priority of main task; two independent task outputs extract electronic signature spatial features according to shared feature representation, identify signature content, and hierarchically classify writing anomalies to detect writing anomalies in real time.

7. The method of claim 6, wherein, The main task output head extracts spatial features in the shared feature representation, and inputs a connection time sequence classification (CTC) unit to decode sequence recognition of the signature content. The CTC unit decoding performs dynamic programming search on the output probability distribution of each time of the signature sequence, selects the character sequence with the highest overall conditional probability, and maps the signature time sequence features to the sequence of the literal content.

8. The method of claim 6, wherein, The auxiliary task output head inputs a multi-layer convolutional neural network layer based on the shared features, performs multi-level feature extraction and down-sampling on the shared features, and extracts hidden features related to abnormal categories in the signature sequence through each network layer. Each hidden feature is subjected to a pooling operation to output significant features reflecting various abnormal patterns. A full connection layer maps these significant features to a discrimination space to achieve accurate classification of the abnormal types. The multi-level feature extraction and down-sampling on the shared features specifically include that a convolutional layer captures changes in writing through a local receptive field, and a pooling layer performs spatial compression on the features to identify parts of shape and structural abnormalities in character writing.

9. The method according to any of claims 6-8, characterized by, According to the formula: The main task output head loss function is calculated, wherein is the target signature text sequence, is the sequence probability predicted by the model; according to the formula: Computing probability distributions for multiple anomaly classes , according to the formula: The loss of the auxiliary task is calculated.

10. The method of claim 9, wherein, Through weighted summation, according to the formula: The loss function of the multi-task joint training module is calculated, wherein, The weight parameter representing the priority of the main task is initially set to ensure that the target of the main task is given priority.

11. An electronic device, comprising: includes: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method of electronic signature sequence content verification and writing abnormality detection based on multi-task learning according to any one of claims 6-10.

12. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein the computer instructions, when executed by a processor, cause the processor to perform the method of any one of claims 1-11. wherein the computer instructions are used to cause the computer to perform the method of electronic signature sequence content verification and writing abnormality detection based on multi-task learning according to any one of claims 6-10.

Images