An electronic signature verification method and device, a computer device and a storage medium

Abstract

The application discloses an electronic signature verification method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. In the application, the identity of a person to be verified is identified in real-time video communication, and after the identity verification is passed, the identity document of the person to be verified is obtained from a document database, and the identity information on the identity document is obtained through OCR identification. In the process of real-time video communication, a handwritten signature image is identified through a handwritten signature recognition model to obtain the identity information on the handwritten signature image. Whether the electronic signature verification is passed is determined by comparing the identity information on the identity document with the identity information on the handwritten signature image. In addition, the application also relates to blockchain technology, and video image data can be stored in the blockchain. The application can ensure the continuity of video connection while completing handwritten signature verification, improve the processing timeliness, and avoid malicious impersonation and signature behavior.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an electronic signature verification method, device, computer equipment, and storage medium. Background Technology

[0002] When insurance companies investigate traffic accidents, they need to prepare an investigation document, which must be signed by the parties involved. When conducting on-site investigations, the investigators verify the identity of the parties involved and witness the authenticity and validity of their signatures. In recent years, remote video conferencing technology has been increasingly applied to insurance accident handling and other business scenarios; however, current electronic signature technology cannot achieve the same effect as on-site investigations.

[0003] Currently, the common practice for electronic signatures is to send a signature link to the party concerned via SMS, and the party opens an H5 link within the SMS to sign. The disadvantages of this method are that it requires redirection to the SMS for signing. If the signature is made during a video call, it disrupts the continuity of the video connection, and some customers may not be able to return to the video call interface. If the signature link is sent after the video call ends, the party concerned may not see the SMS in time, leading to a decrease in case processing efficiency. Furthermore, if someone maliciously impersonates another person to sign, claims personnel may not be able to detect it in a timely manner. Summary of the Invention

[0004] The purpose of this application is to provide an electronic signature verification method, apparatus, computer equipment, and storage medium to solve the technical problem that existing electronic signature verification methods that use SMS links disrupt the continuity of video connections, resulting in low processing efficiency.

[0005] To address the aforementioned technical problems, this application provides an electronic signature verification method, employing the following technical solution:

[0006] An electronic signature verification method includes:

[0007] Receive electronic signature verification instructions and obtain video image data of the person whose signature is to be verified;

[0008] The video image data is analyzed to obtain the facial feature data of the person whose signature is to be verified;

[0009] The facial feature data is imported into a preset identity recognition model to obtain the identity label of the person whose signature is to be verified;

[0010] Based on the identity tag, the identity document of the person whose signature is to be verified is obtained from the preset document database, and the identity document is subjected to OCR recognition to obtain the first identity information;

[0011] Obtain the handwritten signature image of the person whose signature is to be verified, and normalize the handwritten signature image;

[0012] The normalized handwritten signature image is input into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image.

[0013] The shallow feature vector and the deep feature vector are concatenated to obtain a concatenated feature vector.

[0014] The concatenated feature vector is classified and identified using a pre-trained classifier to obtain the second identity information;

[0015] The first identity information and the second identity information are compared to generate an electronic signature verification result.

[0016] Furthermore, the step of parsing the video image data to obtain the facial feature data of the person whose signature is to be verified specifically includes:

[0017] The video image data is parsed, and keyframes are extracted from the parsed video image to obtain keyframe images;

[0018] The keyframe image is identified using a pre-trained content recognition model to obtain the facial image of the person whose signature is to be verified;

[0019] The facial features of the person whose signature is to be verified are obtained by extracting the features of the facial image.

[0020] Furthermore, the step of extracting features from the facial image to obtain the facial feature data of the person whose signature is to be verified specifically includes:

[0021] Construct an image feature extractor;

[0022] The facial features of the person whose signature is to be verified are obtained by using the image feature extractor to extract features from the facial image.

[0023] Furthermore, the identity recognition model includes embedding units, convolutional units, and fully connected units. The step of importing the facial feature data into the preset identity recognition model to obtain the identity label of the person whose signature is to be verified specifically includes:

[0024] The facial feature data is processed by vector transformation through the embedding unit of the identity recognition model to obtain an initial feature vector;

[0025] The initial feature vector is convolved using the convolutional unit of the identity recognition model to obtain the initial feature label;

[0026] The full-connected unit of the identity recognition model is used to calculate the similarity between the initial feature label and the preset feature label, and the feature label corresponding to the maximum similarity is output to obtain the identity label of the person whose signature is to be verified.

[0027] Furthermore, the step of obtaining the handwritten signature image of the person whose signature is to be verified and normalizing the handwritten signature image specifically includes:

[0028] Obtain the handwritten signature image of the person whose signature is to be verified, and perform format normalization processing on the handwritten signature image;

[0029] The handwritten signature image after format normalization is interpolated according to preset standard characters.

[0030] Furthermore, the handwritten signature recognition model includes convolutional layers, pooling layers, and classification layers. The step of inputting the normalized handwritten signature image into the pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image specifically includes:

[0031] Feature extraction is performed on the normalized handwritten signature image to obtain signature feature data;

[0032] The handwritten signature recognition model is used to perform convolution processing on the signature feature data to obtain a shallow feature vector;

[0033] The signature feature data is pooled using the pooling layer of the handwritten signature recognition model to obtain a deep feature vector;

[0034] The classifier is set within the classification layer of the handwritten signature recognition model. The step of using the pre-trained classifier to classify and recognize the concatenated feature vector to obtain the second identity information specifically includes:

[0035] The concatenated feature vector is classified and identified using the classification layer of the handwritten signature recognition model to obtain the second identity information.

[0036] Furthermore, before the step of inputting the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image, the method further includes:

[0037] Obtain sample signature images and annotate them to obtain annotation results;

[0038] Feature extraction is performed on the sample signature image to obtain the sample signature features;

[0039] The sample signature features are imported into a preset initial recognition model, wherein the initial recognition model includes a convolutional layer, a pooling layer, and a classification layer;

[0040] The sample signature features are processed by convolutional layers of the initial recognition model to obtain shallow feature vectors of the sample.

[0041] The pooling layer of the initial recognition model is used to perform pooling processing on the sample signature features to obtain the deep feature vector of the sample;

[0042] The shallow feature vector and the deep feature vector of the sample are concatenated to obtain the concatenated feature vector of the sample.

[0043] The classification layer of the initial recognition model is used to classify and recognize the spliced ​​feature vector of the sample to obtain the recognition result of the sample signature image;

[0044] Based on the recognition results and the annotation results, the initial recognition model is iteratively updated using the backpropagation algorithm until the model is fitted, thus obtaining the handwritten signature recognition model.

[0045] To address the aforementioned technical problems, this application also provides an electronic signature verification device, which employs the following technical solution:

[0046] An electronic signature verification device includes:

[0047] The image data acquisition module is used to receive electronic signature verification instructions and acquire video image data of the person whose signature is to be verified.

[0048] The facial feature extraction module is used to parse the video image data to obtain the facial feature data of the person whose signature is to be verified;

[0049] The identity tag recognition module is used to import the facial feature data into a preset identity recognition model to obtain the identity tag of the person whose signature is to be verified;

[0050] The identity information recognition module is used to obtain the identity document of the person whose signature is to be verified from a preset document database based on the identity tag, and to perform OCR recognition on the identity document to obtain the first identity information;

[0051] The normalization processing module is used to acquire the handwritten signature image of the person whose signature is to be verified, and to perform normalization processing on the handwritten signature image;

[0052] The shallow and deep feature acquisition module is used to input the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image.

[0053] The feature vector concatenation module is used to concatenate the shallow feature vector and the deep feature vector to obtain a concatenated feature vector.

[0054] The feature classification and recognition module is used to classify and recognize the spliced ​​feature vector using a pre-trained classifier to obtain the second identity information.

[0055] The electronic signature verification module is used to compare the first identity information and the second identity information to generate an electronic signature verification result.

[0056] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0057] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the electronic signature verification method as described in any of the preceding claims.

[0058] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0059] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the electronic signature verification method as described in any one of the preceding descriptions.

[0060] Compared with the prior art, the embodiments of this application have the following main advantages:

[0061] This application discloses an electronic signature verification method, apparatus, computer equipment, and storage medium, belonging to the field of artificial intelligence technology. During a real-time video call, this application identifies the identity of the person whose signature is to be verified. After successful identity verification, it retrieves the identity document of the person to be verified from a document database and obtains the identity information on the identity document through OCR recognition. During the real-time video call, it also recognizes the handwritten signature image using a handwritten signature recognition model to obtain the identity information on the handwritten signature image. By comparing the identity information on the identity document with the identity information on the handwritten signature image, it determines whether the electronic signature verification has passed. This application, by combining facial recognition and handwritten signature recognition during a real-time video call to verify the user's handwritten signature information, can ensure the continuity of the video connection while completing handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures. Attached Figure Description

[0062] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 An exemplary system architecture diagram is shown, in which this application can be applied;

[0064] Figure 2 A flowchart of one embodiment of the electronic signature verification method according to this application is shown;

[0065] Figure 3 A schematic diagram of the structure of one embodiment of the electronic signature verification device according to this application is shown;

[0066] Figure 4 A schematic diagram of the structure of one embodiment of a computer device according to this application is shown. Detailed Implementation

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0068] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0069] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0070] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0071] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0072] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0073] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0074] It should be noted that the electronic signature verification method provided in this application embodiment is generally executed by the server, and correspondingly, the electronic signature verification device is generally set in the server.

[0075] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0076] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the electronic signature verification method according to this application. Embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0077] Artificial intelligence (AI) foundational technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning. The aforementioned electronic signature verification method includes the following steps:

[0078] S201: Receive electronic signature verification instructions and obtain video image data of the person whose signature is to be verified.

[0079] Specifically, after receiving the electronic signature verification instruction uploaded by the user, the server initiates a video connection in real time and acquires video image data of the person whose signature is to be verified through the user's camera. The video image data is used to complete facial recognition to obtain the identity label of the person whose signature is to be verified.

[0080] In this embodiment, the electronic signature verification method runs on an electronic device (e.g., Figure 1 The server shown can receive electronic signature verification instructions via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future known wireless connection methods.

[0081] S202, the video image data is analyzed to obtain the facial feature data of the person whose signature is to be verified.

[0082] Specifically, the server parses the video image data to obtain the video image. By extracting key frames from the video image, it obtains key frame images carrying the facial information of the person whose signature is to be verified. Then, it performs content recognition and feature extraction on the key frame images to obtain the facial feature data of the person whose signature is to be verified.

[0083] In order to obtain facial feature data of the person whose signature is to be verified, this application extracts features from the facial image by pre-training an image feature extractor. The image feature extractor is trained based on a special transformer model structure. The facial feature data is obtained by using the pre-trained image feature extractor to extract features from the facial image.

[0084] The Transformer network architecture is a U-shaped neural network architecture. It consists of encoder and decoder layers, which correspond to each other. The feature vectors output by the encoder layer are fed into the corresponding decoder layer for decoding. This U-shaped encoder-decoder network structure allows for the complete extraction of image features across all dimensions of the input image. In addition, a self-attention layer precedes the encoder layer to increase the weights of key features, and a fully connected layer, composed of a softmax function, follows the decoder layer to normalize the results.

[0085] In this application, the image generation network transformer network architecture includes an encoding layer and a decoding layer. The encoding layer has a number of pre-set convolutional kernels, and the decoding layer also has a number of pre-set deconvolutional kernels. Each convolutional kernel corresponds to one deconvolutional kernel. The pre-set convolutional kernels and deconvolutional kernels are untrained initial convolutional kernels and initial deconvolutional kernels. The size of the convolutional kernels and deconvolutional kernels is set according to the feature dimension of the image, such as 128*128, 256*256, etc.

[0086] S203, import the facial feature data into a preset identity recognition model to obtain the identity label of the person whose signature is to be verified.

[0087] The identity recognition model in this application is built upon a convolutional neural network (CNN) structure. CNNs are a type of feedforward neural network with deep structures that incorporate convolutional computations and are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling shift-invariant classification of input information according to their hierarchical structure; therefore, they are also known as "shift-invariant artificial neural networks." CNNs are constructed by mimicking the visual perception mechanisms of biological systems, allowing for both supervised and unsupervised learning. The shared parameters of the convolutional kernels within their convolutional layers and the sparsity of inter-layer connections enable CNNs to learn grid-like topology features, such as pixels and audio, with relatively low computational cost, achieving stable results and requiring no additional feature engineering of the data.

[0088] Specifically, the server pre-trains an identity recognition model based on a CNN convolutional neural network structure. Then, it calculates the similarity between the initial feature label and the preset feature label in the pre-trained identity recognition model, outputs the feature label corresponding to the maximum similarity, and obtains the identity label of the person whose signature is to be verified.

[0089] S204, based on the identity tag, obtain the identity document of the person whose signature is to be verified from the preset document database, and perform OCR recognition on the identity document to obtain the first identity information.

[0090] OCR (Optical Character Recognition) refers to the process by which electronic devices (such as scanners or digital cameras) examine printed characters on paper, determine their shapes by detecting dark and light patterns, and then translate those shapes into computer text using character recognition methods. In other words, for printed characters, it uses optical methods to convert the text in a paper document into a black-and-white dot matrix image file, and then uses recognition software to convert the text in the image into text format for further editing by word processing software. How to correct errors or utilize auxiliary information to improve recognition accuracy is the most important issue in OCR, hence the term ICR (Intelligent Character Recognition). The main indicators for evaluating the performance of an OCR system include: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, ease of use, and feasibility.

[0091] Specifically, the server retrieves the identity document of the person whose signature is to be verified from a pre-set document database based on the identity tag, and performs OCR recognition on the identity document to obtain the information on the identity document, thus obtaining the first identity information. In this process, the person whose signature is to be verified needs to upload their own identity document in advance. For example, in the driving field, drivers need to upload their ID card, driver's license, and other identity documents in advance.

[0092] S205, Obtain the handwritten signature image of the person whose signature is to be verified, and normalize the handwritten signature image.

[0093] The normalization process includes format normalization and character interpolation. Since different users have different writing habits, even the same user will produce different handwritten text in different writing environments. In addition, different writing devices will also affect the recognition of handwritten text. Therefore, it is necessary to perform format normalization and character interpolation on the handwritten signature image to facilitate the subsequent recognition of handwritten signatures.

[0094] Specifically, the system receives a handwritten signature image input by the user and normalizes the image according to the required format. Each character in the handwritten signature image is adjusted to a uniform height and width. Then, each character in the handwritten signature image is interpolated according to a preset standard character to obtain the character trajectory. At the same time, the character trajectory is adjusted to make it easier to recognize in subsequent steps, so that the character trajectory meets the requirements of the standard character. Therefore, image normalization is an essential step for the subsequent recognition of handwritten signatures.

[0095] S206, The normalized handwritten signature image is input into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image.

[0096] Specifically, the handwritten signature recognition model is also built on a convolutional neural network structure. The model includes convolutional layers, pooling layers, and a classification layer. The convolutional layers process the input data to obtain shallow features, the pooling layers process the input data to obtain deep features, and the classification layer classifies the input data based on the shallow and deep features. The server inputs the normalized handwritten signature image into the pre-trained handwritten signature recognition model to obtain the shallow and deep feature vectors of the handwritten signature image.

[0097] S207, The shallow feature vector and the deep feature vector are concatenated to obtain a concatenated feature vector.

[0098] Specifically, after obtaining the shallow feature vector and the deep feature vector, the server concatenates the shallow feature vector and the deep feature vector by concatenating the beginning and end of the vector to obtain the concatenated feature vector.

[0099] S208, the concatenated feature vector is classified and identified using a pre-trained classifier to obtain the second identity information.

[0100] Specifically, a pre-trained classifier is placed in the classification layer of the handwritten signature recognition model, and feature classification can be implemented using the sigmoid function. The server uses the pre-trained classifier to classify and recognize the concatenated feature vectors to obtain the second identity information.

[0101] S209, compare the first identity information and the second identity information to generate an electronic signature verification result.

[0102] Specifically, the server compares the first identity information and the second identity information. When the first identity information and the second identity information match, the server outputs a result indicating that the electronic signature verification has passed. When the first identity information and the second identity information do not match, the server outputs a result indicating that the electronic signature verification has failed and prompts the person whose signature is to be verified to re-verify the signature.

[0103] In the above embodiments, this application verifies the user's handwritten signature information by combining facial recognition and handwritten signature recognition during real-time video calls. This can ensure the continuity of the video connection while completing the handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures.

[0104] Furthermore, the step of parsing the video image data to obtain the facial feature data of the person whose signature is to be verified specifically includes:

[0105] The video image data is parsed, and keyframes are extracted from the parsed video image to obtain keyframe images;

[0106] The keyframe image is identified using a pre-trained content recognition model to obtain the facial image of the person whose signature is to be verified;

[0107] The facial features of the facial image are extracted to obtain the facial feature data of the person whose signature is to be verified.

[0108] Specifically, the server parses the video image data and extracts keyframes from the parsed video image to obtain keyframe images. A pre-trained content recognition model is used to recognize the keyframe images to obtain the facial image of the person whose signature is to be verified. A pre-trained image feature extractor is used to extract the features of the facial image to obtain the facial feature data of the person whose signature is to be verified.

[0109] The server first acquires parameters from two adjacent video frames in the video image and calculates their histogram and grayscale data. The histogram data includes the frame difference values, and the grayscale data includes the mean difference and variance difference. The frame difference values, mean difference, and variance difference of the histogram, grayscale, and grayscale are then weighted and summed to obtain the weighted Euclidean distance between the two adjacent video frames. This weighted Euclidean distance is compared with a preset shot change threshold to determine the shot transition boundary of the action recording video. Finally, keyframe images in the action recording video are determined based on the shot transition boundary.

[0110] It should be noted that before feature extraction, the keyframe image needs to be preprocessed, including sharpening and facial region recognition. Specifically, a pre-defined content recognition model is used to identify the keyframe image, obtaining the human and background contours. The facial region is then identified from the human contour. Next, the keyframe image is denoised to remove salt-and-pepper noise, resulting in a denoised image to ensure image clarity. Based on the denoised image and the facial regions, a clear facial image of the person whose signature is to be verified is obtained.

[0111] In the above embodiments, the video image data is parsed, and keyframes are extracted from the parsed video image to obtain keyframe images. By performing content recognition on the keyframe images, the facial image of the person whose signature is to be verified is obtained. The features of the facial image are extracted by an image feature extractor to obtain the facial feature data of the person whose signature is to be verified.

[0112] Furthermore, the step of extracting features from the facial image to obtain the facial feature data of the person whose signature is to be verified specifically includes:

[0113] Construct an image feature extractor;

[0114] The facial features of the person whose signature is to be verified are obtained by using the image feature extractor to extract features from the facial image.

[0115] Specifically, an image generation network is constructed based on the transformer network architecture. The convolutional kernels and deconvolutional kernels in the image generation network are trained using preset training samples. An image feature extractor is constructed using the trained convolutional kernels and deconvolutional kernels. The image feature extractor is used to extract features from facial images to obtain facial feature data.

[0116] It should be noted that the deep learning compression algorithm filters the trained convolutional and deconvolutional kernels. The deep compression algorithm trains the neural network, obtains the weights of each convolutional layer, sets a weight threshold, and then deletes convolutional layers with weights below the threshold. This process is repeated iteratively to remove redundant layers. Finally, the weights of the remaining convolutional layers are clustered and weights are shared, using the cluster center values ​​as all weight values. By continuously adjusting the cluster centers and their number, a better model compression effect is achieved. Finally, the weights are Huffman encoded. This proposal uses Deep Compression to compress neural networks without sacrificing accuracy, reducing the network size to 35 to 49 times its original size and making storage more efficient during inference.

[0117] In the above embodiments, an image generation network is constructed and trained using training samples and a deep learning compression method to obtain an image feature extractor, which is used to extract facial feature data of the person whose signature is to be verified.

[0118] Furthermore, the identity recognition model includes embedding units, convolutional units, and fully connected units. The step of importing the facial feature data into the preset identity recognition model to obtain the identity label of the person whose signature is to be verified specifically includes:

[0119] The facial feature data is processed by vector transformation through the embedding unit of the identity recognition model to obtain an initial feature vector;

[0120] The initial feature vector is convolved using the convolutional unit of the identity recognition model to obtain the initial feature label;

[0121] The full-connected unit of the identity recognition model is used to calculate the similarity between the initial feature label and the preset feature label, and the feature label corresponding to the maximum similarity is output to obtain the identity label of the person whose signature is to be verified.

[0122] Specifically, the identity recognition model includes an embedding unit, a convolutional unit, and a fully connected unit. The embedding unit performs vector transformation on facial feature data to obtain an initial feature vector. The convolutional unit performs convolution calculations on the initial feature vector to obtain an initial feature label. The fully connected unit calculates the similarity between the initial feature label and a preset feature label, and outputs the feature label corresponding to the maximum similarity, thus obtaining the identity label of the person whose signature is to be verified. Specifically, this application uses a 1:N similarity calculation method to calculate the similarity between the initial feature label and the preset feature label, and outputs the feature label corresponding to the maximum similarity as the identity label of the person whose signature is to be verified.

[0123] It should be noted that the identity recognition model needs to be pre-trained. An initial recognition model is built based on a convolutional neural network structure. Then, labeled training samples are obtained. The training samples are pre-collected images with human faces, and facial features such as forehead, nose, mouth, and eyebrows are pre-annotated on the images. The initial recognition model is then trained using the labeled training samples to obtain training results. The recognition error between the training results and the labeled results is calculated based on the loss function of the initial recognition model. The recognition error is propagated in the initial recognition model using the backpropagation algorithm. The propagated recognition error is compared with a preset error threshold. The initial recognition model is iterated based on the error comparison results. The corresponding model parameters are adjusted in each iteration until the model fits perfectly, thus obtaining the identity recognition model.

[0124] In the above embodiments, this application uses a pre-trained identity recognition model to perform feature extraction, convolution operation and similarity calculation on facial feature data, and finally obtains the identity label of the person whose signature is to be verified.

[0125] Furthermore, the step of obtaining the handwritten signature image of the person whose signature is to be verified and normalizing the handwritten signature image specifically includes:

[0126] Obtain the handwritten signature image of the person whose signature is to be verified, and perform format normalization processing on the handwritten signature image;

[0127] The handwritten signature image after format normalization is interpolated according to preset standard characters.

[0128] Specifically, the process involves acquiring an image of the handwritten signature of the person whose signature is to be verified, performing format normalization on the image, and then interpolating the normalized signature image according to preset standard characters. Format normalization, while maintaining the overall shape of the characters, modifies their size, thus preserving their original appearance and reducing distortion. The character interpolation process uses a specific function to interpolate the characters, employing a linear interpolation constructor to simulate the character trajectory, and then adjusting the trajectory according to the requirements of the standard characters, thereby improving the recognition rate of handwritten text.

[0129] In the above embodiments, this application normalizes the handwritten signature image to make it meet the requirements of standard processing, which facilitates further processing by the subsequent handwritten signature recognition model.

[0130] Furthermore, the handwritten signature recognition model includes convolutional layers, pooling layers, and classification layers. The step of inputting the normalized handwritten signature image into the pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image specifically includes:

[0131] Feature extraction is performed on the normalized handwritten signature image to obtain signature feature data;

[0132] The handwritten signature recognition model is used to perform convolution processing on the signature feature data to obtain a shallow feature vector;

[0133] The signature feature data is pooled using the pooling layer of the handwritten signature recognition model to obtain a deep feature vector;

[0134] The classifier is set within the classification layer of the handwritten signature recognition model. The step of using the pre-trained classifier to classify and recognize the concatenated feature vector to obtain the second identity information specifically includes:

[0135] The concatenated feature vector is classified and identified using the classification layer of the handwritten signature recognition model to obtain the second identity information.

[0136] Specifically, the handwritten signature recognition model also includes an input layer and an output layer. The input layer receives input data and extracts features from it, while the output layer outputs the classification results. The server extracts features from the normalized handwritten signature image to obtain signature feature data. The convolutional layer of the handwritten signature recognition model is used to convolve the signature feature data to obtain a shallow feature vector. The pooling layer of the handwritten signature recognition model is used to pool the signature feature data to obtain a deep feature vector. The classification layer of the handwritten signature recognition model is used to classify and recognize the concatenated feature vector to obtain the second identity information.

[0137] In convolution, the object of the operation is a set of multidimensional matrices. Convolution essentially involves multiplying different local parts of the matrix with the elements of the convolution kernel matrix and then summing the results. However, due to the size limitations of each convolution kernel, convolution operations can only perform shallow feature analysis. Pooling is a method that aggregates features from adjacent locations. Pooled features possess translation and rotation invariance, enabling feature translation processing while reducing the number of features processed and increasing computational efficiency, making it suitable for deep feature analysis. Common pooling operations include average pooling and max pooling, which take the maximum value or average of the corresponding region as the pooled element value, ultimately obtaining the vector matrix corresponding to the deep features of the Chinese signature image. In a convolutional neural network model, the classification layer can be viewed as a specific activation function layer used to classify the information obtained from the previous layer. In this embodiment, the activation function used in the activation function layer is the Sigmoid function. The Sigmoid function is less prone to gradient vanishing during backpropagation, making it easier to train the convolutional neural network model.

[0138] In the above embodiments, this application constructs a handwritten signature recognition model based on a CNN convolutional neural network. The handwritten signature recognition model includes a convolutional layer, a pooling layer, and a classification layer. The convolutional layer is used to perform convolution processing on the input data to obtain shallow features. The pooling layer is used to perform pooling processing on the input data to obtain deep features. The classification layer is used to classify the input data according to the shallow and deep features. By recognizing the handwritten signature image through the handwritten signature recognition model, the second identity information of the person whose signature is to be verified is obtained.

[0139] Furthermore, before the step of inputting the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image, the method further includes:

[0140] Obtain sample signature images and annotate them to obtain annotation results;

[0141] Feature extraction is performed on the sample signature image to obtain the sample signature features;

[0142] The sample signature features are imported into a preset initial recognition model, wherein the initial recognition model includes a convolutional layer, a pooling layer, and a classification layer;

[0143] The sample signature features are processed by convolutional layers of the initial recognition model to obtain shallow feature vectors of the sample.

[0144] The pooling layer of the initial recognition model is used to perform pooling processing on the sample signature features to obtain the deep feature vector of the sample;

[0145] The shallow feature vector and the deep feature vector of the sample are concatenated to obtain the concatenated feature vector of the sample.

[0146] The classification layer of the initial recognition model is used to classify and recognize the spliced ​​feature vector of the sample to obtain the recognition result of the sample signature image;

[0147] Based on the recognition results and the annotation results, the initial recognition model is iteratively updated using the backpropagation algorithm until the model is fitted, thus obtaining the handwritten signature recognition model.

[0148] The backpropagation algorithm, also known as the error backpropagation algorithm (BP algorithm), is a learning algorithm suitable for multi-layer neural networks. It is based on gradient descent and used for error calculation in deep learning networks. The input-output relationship of a BP network is essentially a mapping: a BP neural network with n inputs and m outputs performs a continuous mapping from n-dimensional Euclidean space to a finite field in m-dimensional Euclidean space; this mapping is highly nonlinear. The learning process of the BP algorithm consists of forward propagation and backward propagation. In forward propagation, input information is processed layer by layer through the input layer and hidden layers, and then passed to the output layer, before entering backward propagation. Layer by layer, the partial derivatives of the objective function with respect to the weights of each neuron are calculated, forming the gradient of the objective function with respect to the weight vector, which serves as the basis for modifying the weights.

[0149] Specifically, the process involves acquiring and labeling sample signature images to obtain labeling results; extracting features from the sample signature images to obtain sample signature features; importing the sample signature features into a pre-defined initial recognition model, which includes convolutional layers, pooling layers, and classification layers; using the convolutional layers of the initial recognition model to perform convolution processing on the sample signature features to obtain shallow feature vectors; using the pooling layers of the initial recognition model to perform pooling processing on the sample signature features to obtain deep feature vectors; concatenating the shallow and deep feature vectors to obtain concatenated feature vectors; and using the classification layer of the initial recognition model to classify and recognize the concatenated feature vectors to obtain the recognition result of the sample signature image; based on the recognition and labeling results, using the backpropagation algorithm to iteratively update the initial recognition model until the model is fitted, thus obtaining the handwritten signature recognition model.

[0150] In the above embodiments, this application discloses an electronic signature verification method, belonging to the field of artificial intelligence technology. During a real-time video call, this application identifies the identity of the person whose signature is to be verified. After successful identity verification, it retrieves the identity document of the person whose signature is to be verified from a document database and obtains the identity information on the identity document through OCR recognition. During the real-time video call, it also recognizes the handwritten signature image using a handwritten signature recognition model to obtain the identity information on the handwritten signature image. By comparing the identity information on the identity document with the identity information on the handwritten signature image, it determines whether the electronic signature verification has passed. This application, by combining facial recognition and handwritten signature recognition during a real-time video call to verify the user's handwritten signature information, can ensure the continuity of the video connection while completing the handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures.

[0151] It should be emphasized that, to further ensure the privacy and security of the aforementioned video data, the video data can also be stored in a blockchain node.

[0152] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed, they can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0154] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0155] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an electronic signature verification device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0156] like Figure 3 As shown, the electronic signature verification device described in this embodiment includes:

[0157] The image data acquisition module 301 is used to receive electronic signature verification instructions and acquire video image data of the person whose signature is to be verified.

[0158] The facial feature extraction module 302 is used to parse the video image data to obtain the facial feature data of the person whose signature is to be verified;

[0159] The identity tag recognition module 303 is used to import the facial feature data into a preset identity recognition model to obtain the identity tag of the person whose signature is to be verified.

[0160] The identity information recognition module 304 is used to obtain the identity document of the person whose signature is to be verified from a preset document database based on the identity tag, and to perform OCR recognition on the identity document to obtain the first identity information;

[0161] The normalization processing module 305 is used to acquire the handwritten signature image of the person whose signature is to be verified, and to perform normalization processing on the handwritten signature image;

[0162] The shallow and deep feature acquisition module 306 is used to input the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image.

[0163] The feature vector concatenation module 307 is used to concatenate the shallow feature vector and the deep feature vector to obtain a concatenated feature vector.

[0164] The feature classification and recognition module 308 is used to classify and recognize the spliced ​​feature vector using a pre-trained classifier to obtain the second identity information.

[0165] The electronic signature verification module 309 is used to compare the first identity information and the second identity information to generate an electronic signature verification result.

[0166] Furthermore, the facial feature extraction module 302 specifically includes:

[0167] The keyframe extraction unit is used to parse the video image data and extract keyframes from the parsed video image to obtain keyframe images.

[0168] The content recognition unit is used to recognize the keyframe image using a pre-trained content recognition model to obtain the facial image of the person whose signature is to be verified.

[0169] A facial feature extraction unit is used to extract features from the facial image to obtain facial feature data of the person whose signature is to be verified.

[0170] Furthermore, the facial feature extraction unit specifically includes:

[0171] The extractor building subunit is used to build the image feature extractor;

[0172] The facial feature extraction subunit is used to extract features from the facial image using the image feature extractor to obtain the facial feature data of the person whose signature is to be verified.

[0173] Furthermore, the identity recognition model includes embedding units, convolutional units, and fully connected units, and the identity tag recognition module 303 specifically includes:

[0174] The vector transformation unit is used to perform vector transformation processing on the facial feature data through the embedding unit of the identity recognition model to obtain an initial feature vector;

[0175] A convolutional computation unit is used to perform convolutional computation on the initial feature vector using the convolutional unit of the identity recognition model to obtain initial feature labels;

[0176] The similarity calculation unit is used to calculate the similarity between the initial feature label and the preset feature label using the fully connected unit of the identity recognition model, and output the feature label corresponding to the maximum similarity value to obtain the identity label of the person whose signature is to be verified.

[0177] Furthermore, the normalization processing module 305 specifically includes:

[0178] The format normalization unit is used to acquire the handwritten signature image of the person whose signature is to be verified, and to perform format normalization processing on the handwritten signature image;

[0179] An interpolation processing unit is used to perform interpolation processing on the handwritten signature image after format normalization according to preset standard characters.

[0180] Furthermore, the handwritten signature recognition model includes convolutional layers, pooling layers, and classification layers, and the deep / shallow feature acquisition module 306 specifically includes:

[0181] The signature feature extraction unit is used to extract features from the normalized handwritten signature image to obtain signature feature data.

[0182] The convolution processing unit is used to perform convolution processing on the signature feature data using the convolutional layer of the handwritten signature recognition model to obtain a shallow feature vector.

[0183] The pooling processing unit is used to perform pooling processing on the signature feature data using the pooling layer of the handwritten signature recognition model to obtain a deep feature vector.

[0184] The classifier is set within the classification layer of the handwritten signature recognition model. The step of using the pre-trained classifier to classify and recognize the concatenated feature vector to obtain the second identity information specifically includes:

[0185] The classification and recognition unit is used to classify and recognize the spliced ​​feature vector using the classification layer of the handwritten signature recognition model to obtain the second identity information.

[0186] Furthermore, the electronic signature verification device also includes:

[0187] The sample annotation module is used to acquire sample signature images and annotate the sample signature images to obtain annotation results;

[0188] The sample feature extraction module is used to extract features from the sample signature image to obtain sample signature features;

[0189] The sample data import module is used to import the sample signature features into a preset initial recognition model, wherein the initial recognition model includes a convolutional layer, a pooling layer, and a classification layer.

[0190] The sample convolution module is used to perform convolution processing on the sample signature features using the convolutional layer of the initial recognition model to obtain the sample shallow feature vector;

[0191] The sample pooling module is used to perform pooling processing on the sample signature features using the pooling layer of the initial recognition model to obtain the deep feature vector of the sample.

[0192] The sample vector concatenation module is used to concatenate the shallow feature vector and the deep feature vector of the sample to obtain the concatenated feature vector.

[0193] The sample signature recognition module is used to classify and recognize the concatenated feature vector of the sample using the classification layer of the initial recognition model, so as to obtain the recognition result of the sample signature image.

[0194] The recognition model iteration module is used to iteratively update the initial recognition model based on the recognition results and the annotation results using the backpropagation algorithm until the model is fitted, thereby obtaining the handwritten signature recognition model.

[0195] In the above embodiments, this application discloses an electronic signature verification device, belonging to the field of artificial intelligence technology. During real-time video calls, this application identifies the identity of the person whose signature is to be verified. After successful identity verification, it retrieves the identity document of the person whose signature is to be verified from a document database and obtains the identity information on the identity document through OCR recognition. During the real-time video call, it also recognizes the handwritten signature image using a handwritten signature recognition model to obtain the identity information on the handwritten signature image. By comparing the identity information on the identity document with the identity information on the handwritten signature image, it determines whether the electronic signature verification has passed. This application, by combining facial recognition and handwritten signature recognition during real-time video calls to verify the user's handwritten signature information, can ensure the continuity of the video connection while completing handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures.

[0196] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0197] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0198] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0199] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for electronic signature verification methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0200] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, such as executing computer-readable instructions for the electronic signature verification method.

[0201] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0202] This application discloses a computer device belonging to the field of artificial intelligence technology. During a real-time video call, this application identifies the identity of the person whose signature is to be verified. After successful identity verification, it retrieves the identity document of the person to be verified from a document database and obtains the identity information on the identity document through OCR recognition. During the real-time video call, it also recognizes the handwritten signature image using a handwritten signature recognition model to obtain the identity information on the handwritten signature image. By comparing the identity information on the identity document with the identity information on the handwritten signature image, it determines whether the electronic signature verification has passed. This application, by combining facial recognition and handwritten signature recognition during a real-time video call to verify the user's handwritten signature information, can ensure the continuity of the video connection while completing handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures.

[0203] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to perform the steps of the electronic signature verification method described above. The execution of the electronic signature verification method by the at least one processor achieves the following technical effects:

[0204] During a real-time video call, the identity of the person whose signature is to be verified is identified. After successful identity verification, the identity document of the person to be verified is retrieved from the document database, and the identity information on the identity document is obtained through OCR recognition. During the real-time video call, the handwritten signature image is recognized using a handwritten signature recognition model to obtain the identity information on the handwritten signature image. By comparing the identity information on the identity document with the identity information on the handwritten signature image, it is determined whether the electronic signature verification has passed. This application, by combining facial recognition and handwritten signature recognition during a real-time video call to verify the user's handwritten signature information, can ensure the continuity of the video connection while completing the handwritten signature verification, improve processing efficiency, and prevent malicious impersonation of others' signatures.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0206] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0207] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. An electronic signature verification method, characterized in that, include: Receive electronic signature verification instructions and obtain video image data of the person whose signature is to be verified; The video image data is analyzed to obtain the facial feature data of the person whose signature is to be verified; The facial feature data is imported into a preset identity recognition model to obtain the identity label of the person whose signature is to be verified; Based on the identity tag, the identity document of the person whose signature is to be verified is obtained from the preset document database, and the identity document is subjected to OCR recognition to obtain the first identity information; The handwritten signature image of the person whose signature is to be verified is obtained, and the handwritten signature image is normalized. The normalization process includes format normalization and character interpolation. The format normalization process performs format normalization on the handwritten signature image, and then performs interpolation on the format normalized handwritten signature image according to preset standard characters. The format normalization process changes the size of the characters while keeping the overall shape of the characters unchanged. The character interpolation process performs interpolation on the characters according to a specific function and uses a linear interpolation constructor to simulate the character trajectory. Then, the character trajectory is adjusted according to the requirements of the standard characters. The normalized handwritten signature image is input into a pre-trained handwritten signature recognition model to obtain shallow and deep feature vectors of the handwritten signature image. The handwritten signature recognition model includes convolutional layers, pooling layers, and classification layers. Features are extracted from the normalized handwritten signature image to obtain signature feature data. The signature feature data is then convolved using the convolutional layers of the handwritten signature recognition model to obtain shallow feature vectors. Finally, the signature feature data is pooled using the pooling layers of the handwritten signature recognition model to obtain deep feature vectors. The shallow feature vector and the deep feature vector are concatenated to obtain a concatenated feature vector. The concatenated feature vector is classified and identified using a pre-trained classifier to obtain second identity information. The classifier is set in the classification layer of the handwritten signature recognition model. The concatenated feature vector is classified and identified using the classification layer of the handwritten signature recognition model to obtain second identity information. The first identity information and the second identity information are compared to generate an electronic signature verification result. The identity recognition model includes an embedding unit, a convolutional unit, and a fully connected unit. The embedding unit of the identity recognition model performs vector transformation processing on the facial feature data to obtain an initial feature vector. The convolutional unit of the identity recognition model performs convolution calculation on the initial feature vector to obtain an initial feature label. The fully connected unit of the identity recognition model calculates the similarity between the initial feature label and a preset feature label, and outputs the feature label corresponding to the maximum similarity to obtain the identity label of the person whose signature is to be verified.

2. The electronic signature verification method as described in claim 1, characterized in that, The step of parsing the video image data to obtain the facial feature data of the person whose signature is to be verified specifically includes: The video image data is parsed, and keyframes are extracted from the parsed video image to obtain keyframe images; The keyframe image is identified using a pre-trained content recognition model to obtain the facial image of the person whose signature is to be verified; The facial features of the facial image are extracted to obtain the facial feature data of the person whose signature is to be verified.

3. The electronic signature verification method as described in claim 2, characterized in that, The step of extracting features from the facial image to obtain the facial feature data of the person whose signature is to be verified specifically includes: Construct an image feature extractor; The facial features of the person whose signature is to be verified are obtained by using the image feature extractor to extract features from the facial image.

4. The electronic signature verification method as described in claim 1, characterized in that, Before the step of inputting the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow and deep feature vectors of the handwritten signature image, the method further includes: Obtain sample signature images and annotate them to obtain annotation results; Feature extraction is performed on the sample signature image to obtain the sample signature features; The sample signature features are imported into a preset initial recognition model, wherein the initial recognition model includes a convolutional layer, a pooling layer, and a classification layer; The sample signature features are processed by convolutional layers of the initial recognition model to obtain shallow feature vectors of the sample. The pooling layer of the initial recognition model is used to perform pooling processing on the sample signature features to obtain the deep feature vector of the sample; The shallow feature vector and the deep feature vector of the sample are concatenated to obtain the concatenated feature vector of the sample. The classification layer of the initial recognition model is used to classify and recognize the spliced ​​feature vector of the sample to obtain the recognition result of the sample signature image; Based on the recognition results and the annotation results, the initial recognition model is iteratively updated using the backpropagation algorithm until the model is fitted, thus obtaining the handwritten signature recognition model.

5. An electronic signature verification device, characterized in that, The electronic signature verification device performs the steps of the electronic signature verification method as described in any one of claims 1 to 4, and the electronic signature verification device includes: The image data acquisition module is used to receive electronic signature verification instructions and acquire video image data of the person whose signature is to be verified. The facial feature extraction module is used to parse the video image data to obtain the facial feature data of the person whose signature is to be verified; The identity tag recognition module is used to import the facial feature data into a preset identity recognition model to obtain the identity tag of the person whose signature is to be verified; The identity information recognition module is used to obtain the identity document of the person whose signature is to be verified from a preset document database based on the identity tag, and to perform OCR recognition on the identity document to obtain the first identity information; The normalization processing module is used to acquire the handwritten signature image of the person whose signature is to be verified, and to perform normalization processing on the handwritten signature image; The shallow and deep feature acquisition module is used to input the normalized handwritten signature image into a pre-trained handwritten signature recognition model to obtain the shallow feature vector and deep feature vector of the handwritten signature image. The feature vector concatenation module is used to concatenate the shallow feature vector and the deep feature vector to obtain a concatenated feature vector. The feature classification and recognition module is used to classify and recognize the spliced ​​feature vector using a pre-trained classifier to obtain the second identity information. The electronic signature verification module is used to compare the first identity information and the second identity information to generate an electronic signature verification result.

6. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the electronic signature verification method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the electronic signature verification method as described in any one of claims 1 to 4.

Images