Training method, data processing method, device, storage medium and program product

Abstract

The application provides a training method, a data processing method, equipment, a storage medium and a computer program product, which are applied to various information verification scenes such as cloud technology, artificial intelligence, intelligent transportation and vehicle-mounted devices; the method comprises the following steps: obtaining a training sample, wherein the training sample comprises a label image, a sample image, a signature label of the sample image, and a similarity label between the label image and the sample image; based on a to-be-trained model, the following processing is performed: extracting a first sample feature of the sample image and a second sample feature of the label image, performing signature identification based on the first sample feature to obtain signature identification information, and determining similarity prediction information based on the first sample feature and the second sample feature; combining the difference between the signature identification information and the signature label and the difference between the similarity prediction information and the similarity label, training the to-be-trained model, and obtaining a signature verification model. Through the application, the model training efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer applications, in particular to a training method, a data processing method, equipment, a storage medium and a program product. BACKGROUND

[0002] An electronic handwritten signature refers to data for identifying the identity of a signature object in an electronic form, usually in the form of an image, and is different from a digital image of a handwritten signature.

[0003] In the process of using an electronic handwritten signature, verification of the electronic handwritten signature is often involved; generally, verification of the electronic handwritten signature is implemented by a text recognition model and an image similarity calculation model; since the text recognition model and the image similarity calculation model are two independent models and are obtained by independent training, the training efficiency of the models is affected. SUMMARY

[0004] The embodiments of the present application provide a training method of a signature verification model, a data processing method based on signature verification, an apparatus, a device, a computer readable storage medium and a computer program product, which can improve the training efficiency of the model.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide a training method of a signature verification model, which comprises the following steps:

[0007] Obtaining a training sample, wherein the training sample comprises a label image, a sample image, a signature label of the sample image, and a similarity label between the label image and the sample image, and the label image and the sample image are both electronic handwritten signature images;

[0008] Based on a to-be-trained model, the following processing is performed: extracting a first sample feature of the sample image and a second sample feature of the label image, performing signature recognition based on the first sample feature to obtain signature recognition information, and determining similarity prediction information based on the first sample feature and the second sample feature;

[0009] Training the to-be-trained model based on the difference between the signature recognition information and the signature label and the difference between the similarity prediction information and the similarity label, to obtain the signature verification model, wherein the signature verification model is used for verifying an electronic handwritten signature of a to-be-verified signature image

[0010] The embodiments of the present application provide a data processing method based on signature verification, which comprises the following steps:

[0011] verify the electronic handwritten signature of the to-be-verified signature image based on a signature verification model, to obtain target signature text of the to-be-verified signature image and target similarity between the to-be-verified signature image and the standard signature image, wherein the signature verification model is obtained by training the signature verification model according to the training method of the signature verification model provided in the embodiments of the present application;

[0012] determine a verification result of the to-be-verified signature image based on a matching result between the target signature text and the standard signature text and a comparison result between the target similarity and a similarity threshold.

[0013] The embodiments of the present application provide a training device of a signature verification model, comprising:

[0014] a sample obtaining module configured to obtain training samples, wherein the training samples comprise a label image, a sample image, a signature label of the sample image, and a similarity label between the label image and the sample image, and the label image and the sample image are both electronic handwritten signature images;

[0015] a model verifying module configured to perform the following processing based on a to-be-trained model: extract a first sample feature of the sample image and a second sample feature of the label image, perform signature recognition based on the first sample feature to obtain signature recognition information, and determine similarity prediction information based on the first sample feature and the second sample feature;

[0016] a model training module configured to train the to-be-trained model based on a difference between the signature recognition information and the signature label and a difference between the similarity prediction information and the similarity label, to obtain the signature verification model, wherein the signature verification model is used to verify the electronic handwritten signature of a to-be-verified signature image.

[0017] In the embodiments of the present application, the model training module is further configured to fuse the difference between the signature recognition information and the signature label with a first difference weight to obtain a first target difference, fuse the difference between the similarity prediction information and the similarity label with a second difference weight to obtain a second target difference, and train the to-be-trained model based on the first target difference and the second target difference to obtain the signature verification model.

[0018] In the embodiment of the present application, the model training module is further configured to: obtain a test sample during training of the to-be-trained model; perform verification of electronic handwritten signatures on the test sample based on the to-be-trained model currently trained to obtain a signature accuracy rate and a similarity accuracy rate; when the signature accuracy rate is greater than the similarity accuracy rate, perform first adjustment on at least one of the first difference weight and the second difference weight to obtain a first target difference weight smaller than a second target difference weight; when the signature accuracy rate is less than the similarity accuracy rate, perform second adjustment on at least one of the first difference weight and the second difference weight to obtain a first target difference weight greater than the second target difference weight; and obtain a target loss used for training the to-be-trained model currently trained based on the first target difference weight and the second target difference weight.

[0019] In the embodiment of the present application, the to-be-trained model comprises a feature extraction module, the feature extraction module comprises a first sub-feature extraction module and a second sub-feature extraction module, and the first sub-feature extraction module and the second sub-feature extraction module share model parameters; and the model training module is further configured to extract the first sample feature of the sample image through the first sub-feature extraction module and extract the second sample feature of the label image through the second sub-feature extraction module.

[0020] In the embodiment of the present application, the signature label comprises at least one of a signature position label and a signature content label, when the signature label comprises the signature position label and the signature content label, the model training module is further configured to: perform signature position detection based on the first sample feature to obtain position prediction information; and perform signature content recognition based on the first sample feature to obtain content prediction information, wherein the signature recognition comprises the signature position detection and the signature content recognition; and determine the position prediction information and the content prediction information as the signature recognition information.

[0021] In the embodiment of the present application, the model training module is further configured to combine a difference between the position prediction information and the signature position label and a difference between the content prediction information and the signature content label as a difference between the signature recognition information and the signature label.

[0022] In the embodiment of the present application, the model training module is further configured to: the sample image comprises at least one of a positive sample image and a negative sample image, wherein the positive sample image and the label image are derived from the same signature object, and the negative sample image and the label image are derived from different signature objects.

[0023] In the embodiment of the present application, when the sample image includes a positive sample image and a negative sample image, the first sample feature includes a first sub-sample feature of the positive sample image and a second sub-sample feature of the negative sample image, and the model training module is further configured to: acquire a first feature distance between the first sub-sample feature and the second sample feature; acquire a second feature distance between the second sub-sample feature and the second sample feature; and determine the first feature distance and the second feature distance as the similarity prediction information.

[0024] In the embodiment of the present application, the model training module is further configured to: acquire a feature distance difference between the first feature distance and the second feature distance based on the similarity label; and determine a difference between the similarity prediction information and the similarity label based on a comparison result of the feature distance difference and a feature distance threshold.

[0025] In the embodiment of the present application, the model training module is further configured to acquire a new training sample, wherein the new training sample includes a new label image, a new sample image, a new signature label of the new sample image, and a new similarity label between the new label image and the new sample image, and the new label image and the new sample image are both electronic handwritten signature images; and perform the following processing based on the signature verification model: extracting a first new sample feature of the new sample image and a second new sample feature of the new label image, performing signature recognition on the first new sample feature to obtain new signature recognition information, and determining new similarity prediction information based on the first new sample feature and the second new sample feature; and combining a difference between the new signature recognition information and the new signature label and a difference between the new similarity prediction information and the new similarity label, optimizing the signature verification model to obtain a target signature verification model, wherein the target signature verification model is used for verifying a new to-be-verified signature image in electronic handwritten signature verification.

[0026] The embodiment of the present application provides a data processing apparatus based on signature verification, comprising:

[0027] A signature verification module is configured to perform electronic handwritten signature verification on a to-be-verified signature image and a standard signature image based on a signature verification model to obtain target signature text of the to-be-verified signature image and target similarity between the to-be-verified signature image and the standard signature image, wherein the signature verification model is obtained by training through the training method provided in the embodiment of the present application.

[0028] A result processing module is configured to combine a matching result between the target signature text and standard signature text and a comparison result between the target similarity and a similarity threshold to determine a verification result of the to-be-verified signature image.

[0029] In the embodiment of the present application, the data processing apparatus further comprises an initialization module configured to obtain an initialization signature image and an initialization signature text; perform signature recognition on the initialization signature image based on the signature verification model to obtain a signature text recognition result; and determine the initialization signature image as the standard signature image and determine the initialization signature text as the standard signature text when the signature text recognition result matches the standard signature text.

[0030] In the embodiment of the present application, the result processing module is further configured to display a signature interface in response to a triggering operation on a verification control, wherein the verification control is configured to trigger acquisition of the signature image to be verified; and obtain the signature image to be verified in response to a signature operation on the signature interface.

[0031] In the embodiment of the present application, the result processing module is further configured to perform target processing when the verification result indicates that the target signature text matches the standard signature text and the target similarity is greater than the similarity threshold, wherein the target processing comprises at least one of account login, information update, and account state unblocking.

[0032] An embodiment of the present application provides a training device, comprising:

[0033] A first memory configured to store computer executable instructions;

[0034] A first processor configured to execute the computer executable instructions stored in the first memory to implement the training method of the signature verification model provided in the embodiments of the present application.

[0035] An embodiment of the present application provides a training device, comprising:

[0036] A second memory configured to store computer executable instructions;

[0037] A second processor configured to execute the computer executable instructions stored in the second memory to implement the data processing method based on signature verification provided in the embodiments of the present application.

[0038] An embodiment of the present application provides a computer readable storage medium storing computer executable instructions, wherein the computer executable instructions are configured to be executed by a first processor to implement the training method of the signature verification model provided in the embodiments of the present application, or the computer executable instructions are configured to be executed by a second processor to implement the data processing method based on signature verification provided in the embodiments of the present application.

[0039] The embodiment of the application provides a computer program product, comprising a computer program or computer executable instructions, which, when executed by a first processor, implement the training method of the signature verification model provided by the embodiment of the application; or, when executed by a second processor, implement the data processing method based on signature verification provided by the embodiment of the application.

[0040] The embodiment of the application has at least the following beneficial effects: for the to-be-trained model capable of realizing signature recognition and similarity prediction, by acquiring a training sample and training the to-be-trained model signature recognition and similarity prediction by using the training sample, the overall training of the signature recognition and the image similarity prediction two tasks in one model is realized, the complexity of model training is reduced, and therefore, the model training efficiency can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0042] Figure 2 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0043] Figure 3 is an exemplary schematic diagram of verifying an electronic handwritten signature; Figure 2 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0044] Figure 4 is an exemplary schematic diagram of verifying an electronic handwritten signature; Figure 2 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0045] Figure 5 is an exemplary schematic diagram of verifying an electronic handwritten signature; Figure 1 ;

[0046] Figure 6 is an exemplary schematic diagram of verifying an electronic handwritten signature; Figure 2 ;

[0047] Figure 7 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0048] Figure 8 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0049] Figure 9 is an exemplary schematic diagram of verifying an electronic handwritten signature;

[0050] Figure 10is a structural schematic diagram of an exemplary text recognition module provided by an embodiment of the present application.

[0051] Figure 11 is an exemplary device interaction schematic diagram provided by an embodiment of the present application.

[0052] Figure 12 is an exemplary subsequent processing scenario schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0054] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0055] In the following description, the terms "first\second" are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that "first\second" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0056] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application are the same as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0057] The related data collection processing in the embodiments of the present application should be strictly in accordance with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and within the scope of authorization of laws and regulations and the personal information subject, carry out subsequent data use and processing.

[0058] Before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations.

[0059] 1) Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0060] 2) Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis and algorithm complexity theory. It is used to study how computers simulate or implement human learning behavior to acquire new knowledge or skills; reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning and inductive learning.

[0061] 3) Artificial neural network is a mathematical model that simulates the structure and function of biological neural network. Exemplary structures include Graph Convolutional Network (GCN, a neural network for processing graph-structured data), Deep Neural Networks (DNN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), Neural State Machine (NSM) and Phase-Functioned Neural Network (PFNN). The signature verification model and the model to be trained in the embodiments of the present application are all corresponding models of artificial neural network.

[0062] It should be noted that in the process of using electronic handwritten signature, the verification of electronic handwritten signature is often involved; generally, the verification of electronic handwritten signature is realized through a text recognition model and an image similarity calculation model; since the text recognition model and the image similarity calculation model are two independent models, they are obtained by independent training, which affects the training efficiency of the model.

[0063] Exemplarily, refer to Figure 1 , Figure 1 is an exemplary schematic diagram for verifying electronic handwritten signature; as Figure 1As shown, when starting to verify the electronic handwritten signature, first, the electronic handwritten signature image 1-1 to be verified is acquired; the electronic handwritten signature image 1-1 is recognized by using the text recognition model 1-2 to obtain the recognized name information 1-3, and the initialized name information 1-4 and the name information 1-3 are compared; the similarity of the electronic handwritten signature image 1-1 and the initialized electronic handwritten signature image 1-6 is calculated by using the image similarity calculation model 1-5; thus, the verification result 1-7 is determined in combination with the name information comparison result and the similarity comparison result, and the verification of the electronic handwritten signature is ended. Here, since the verification of the electronic handwritten signature is realized by two independent models (the text recognition model 1-2 and the image similarity calculation model 1-5), training the two independent models affects the training efficiency of the model and also affects the accuracy of the model; in addition, in the process of verifying the electronic handwritten signature by using two independent models to respectively perform text recognition and image similarity calculation, the use of the two models leads to high complexity of the electronic handwritten signature verification, and thus the verification efficiency of the electronic handwritten signature is affected.

[0064] Based on this, the embodiments of the present application provide a signature verification model training method, a signature verification-based data processing method and device, a computer readable storage medium and a computer program product, which can improve the training efficiency of the model and the accuracy of the model. The following describes an exemplary application of the device provided by the embodiments of the present application. The device provided by the embodiments of the present application can be implemented as various types of terminals such as a smartphone, a smart watch, a notebook computer, a tablet computer, a desktop computer, a smart home appliance, a set-top box, a smart vehicle device, a portable music player, a personal digital assistant, a dedicated message device, a smart voice interaction device, a portable game device, and a smart speaker, and can also be implemented as a server. The following describes an exemplary application when the training device is implemented as a server and the data processing device is implemented as a terminal.

[0065] Referring to Figure 2 , Figure 2 is an architecture schematic diagram of a signature verification system provided by the embodiments of the present application; as Figure 2 shown, to support a signature verification application, in the signature verification system 100, the terminal 400 (exemplarily shown as the terminal 400-1 and the terminal 400-2, referred to as a data processing device) is connected to the server 200 (referred to as a training device) through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two. In addition, the signature verification system 100 further includes the database 500, which is used to provide data support to the server 200; and Figure 2 shown in the figure is a case where the database 500 is independent of the server 200. In addition, the database 500 can also be integrated in the server 200, which is not limited in the embodiments of the present application.

[0066] The terminal 400 is configured to receive the signature verification model sent by the server 200 through the network 300, deploy the signature verification model, and perform electronic handwritten signature verification on the to-be-verified signature image and the standard signature image by using the deployed signature verification model (exemplarily shown as the graphical interface 410-1 and the graphical interface 410-2), to obtain the target signature text of the to-be-verified signature image and the target similarity between the to-be-verified signature image and the standard signature image; determine the verification result of the to-be-verified signature image by combining the matching result between the target signature text and the standard signature text and the comparison result between the target similarity and the similarity threshold. The terminal 400 is also configured to send the to-be-verified signature image to the server 200 through the network 300, receive the verification result sent by the server 200 for the to-be-verified signature image, and display the verification result on the graphical interface.

[0067] The server 200 is configured to obtain training samples, wherein the training samples include label images, sample images, signature labels of the sample images, and similarity labels between the label images and the sample images, and the label images and the sample images are both electronic handwritten signature images; perform the following processing based on a to-be-trained model: extract first sample features of the sample images and second sample features of the label images, perform signature recognition based on the first sample features to obtain signature recognition information, and determine similarity prediction information based on the first sample features and the second sample features; train the to-be-trained model by combining the difference between the signature recognition information and the signature labels and the difference between the similarity prediction information and the similarity labels, to obtain a signature verification model; and send the signature verification model to the terminal 400 through the network 300. The server 200 is also configured to receive the to-be-verified signature image sent by the terminal 400 through the network 300, and perform electronic handwritten signature verification on the to-be-verified signature image and the standard signature image based on the signature verification model, to obtain the target signature text of the to-be-verified signature image and the target similarity between the to-be-verified signature image and the standard signature image; determine the verification result of the to-be-verified signature image by combining the matching result between the target signature text and the standard signature text and the comparison result between the target similarity and the similarity threshold; and send the verification result to the terminal 400 through the network 300.

[0068] It should be noted that the model training and the model application can be implemented by the same server on the server 200, in which case the server 200 can be one server or multiple servers; or the model training and the model application can be implemented by different servers on the server 200, in which case the server 200 is multiple servers.

[0069] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, smartwatch, laptop, tablet, desktop computer, smart TV, set-top box, smart in-vehicle device, portable music player, personal digital assistant, dedicated messaging device, portable gaming device, and smart speaker, but is not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0070] See Figure 3 , Figure 3 This is provided by the embodiments of this application. Figure 2 A schematic diagram of the composition structure of a server. Figure 3 The server 200 shown includes the following components: at least one first processor 210, a first memory 250, and at least one first network interface 220. The components of server 200 are coupled together via a first bus system 240. It is understood that the first bus system 240 is used to implement communication between these components. In addition to a data bus, the first bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general designated all buses as the first bus system 240.

[0071] The first processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0072] The first memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The first memory 250 may optionally include one or more storage devices physically located remote from the first processor 210.

[0073] The first memory 250 includes volatile memory or non-volatile memory, and can include both volatile and non-volatile memory. The non-volatile memory can be read only memory (ROM), and the volatile memory can be random access memory (RAM). The first memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0074] In some embodiments, the first memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are exemplarily explained below.

[0075] The first operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks;

[0076] The first network communication module 252 is used to reach other computer devices via one or more (wired or wireless) first network interfaces 220, and exemplary first network interfaces 220 include Bluetooth, wireless compatibility authentication (Wi-Fi), universal serial bus (USB), and the like;

[0077] In some embodiments, the training device provided by the embodiments of the present application can be implemented in a software manner, Figure 3 The training device 255 stored in the first memory 250 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a sample acquisition module 2551, a model verification module 2552, and a model training module 2553. These modules are logical, and thus can be combined or further split according to the functions implemented. The functions of each module will be explained below.

[0078] In some embodiments, the training device provided in this application can be implemented in hardware. As an example, the training device provided in this application can be a first processor in the form of a hardware decoding processor, which is programmed to execute the training method of the signature verification model provided in this application. For example, the first processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0079] See Figure 4 , Figure 4 This is provided by the embodiments of this application. Figure 2 A schematic diagram of the structural composition of one type of terminal. Figure 4 The terminal 400 shown includes at least one second processor 410, a second memory 450, at least one second network interface 420, and a second user interface 430. The various components in the terminal 400 are coupled together via a second bus system 440. It is understood that the second bus system 440 is used to implement communication between these components. In addition to a data bus, the second bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The various buses are all labeled as the second bus system 440.

[0080] The second processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0081] The second user interface 430 includes one or more second output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. The second user interface 430 also includes one or more second input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0082] The second memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The second memory 450 may optionally include one or more storage devices physically located remote from the second processor 410.

[0083] The second memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory, and the volatile memory may be random access memory. The second memory 450 described in the embodiments of this application is intended to include any suitable type of memory.

[0084] In some embodiments, the second memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0085] The second operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0086] The second network communication module 452 is used to reach other computer devices via one or more (wired or wireless) second network interfaces 420. Exemplary second network interfaces 420 include Bluetooth, wireless compatibility authentication, and Universal Serial Bus (USB), etc.

[0087] The second presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more second output devices 431 (e.g., a display screen, a speaker, etc.) associated with the second user interface 430.

[0088] The second input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more second input devices 432.

[0089] In some embodiments, the data processing apparatus provided in this application can be implemented in software. Figure 4 A data processing device 455 stored in a second memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a signature verification module 4551, a result processing module 4552, and an initialization module 4553. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0090] In some embodiments, the data processing apparatus provided in this application can be implemented in hardware. As an example, the data processing apparatus provided in this application can be a second processor in the form of a hardware decoding processor, which is programmed to execute the signature verification-based data processing method provided in this application. For example, the second processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits, DSPs, programmable logic devices, complex programmable logic devices, field-programmable gate arrays or other electronic components.

[0091] In some embodiments, the terminal or server can implement the signature verification-based data processing method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a live streaming APP or an instant messaging APP; it can also be a mini-program, that is, a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0092] The following will describe the training method for the signature verification model and the data processing method based on signature verification provided in this application, using exemplary applications and implementations of the training equipment and data processing equipment provided in the embodiments of this application. Furthermore, the training method for the signature verification model and the data processing method based on signature verification provided in the embodiments of this application can be applied to various information verification scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems.

[0093] See Figure 5 , Figure 5 This is a flowchart illustrating the training method of the signature verification model provided in the embodiments of this application. Figure 1 , will combine Figure 5 The steps shown are explained.

[0094] Step 501: Obtain training samples, wherein the training samples include label images, sample images, signature labels of sample images, and similar labels between label images and sample images.

[0095] In this embodiment of the application, when the training device performs model training, it acquires data for training the model, thus obtaining training samples. The process of acquiring training samples includes: acquiring a training image set, calibrating the training image set, and combining the training image set with the calibration results to form training samples. The training image set acquired by the training device includes E electronic handwritten signature images corresponding to M signature objects (e.g., users, accounts, agents, etc.), where E is a positive integer greater than 1, and M is a positive integer. Furthermore, the label image and sample image are selected from E electronic handwritten signature images, thus both the label image and the sample image are electronic handwritten signature images; the calibration results include the signature label of the sample image, as well as the similarity labels between the label image and the sample image.

[0096] It should be noted that an electronic handwritten signature image refers to an image containing an electronic handwritten signature. A signature label refers to the identification information of the electronic handwritten signature in the sample image, including at least one of the location and content of the electronic handwritten signature. A similarity label indicates whether a label image and a sample image are similar; specifically, when the label image and the sample image are electronic handwritten signatures of the same signature object, the similarity label indicates that the label image and the sample image are similar, that is, the electronic handwritten signature in the label image is similar to the electronic handwritten signature in the sample image, for example, represented by "1"; when the label image and the sample image are electronic handwritten signatures of different signature objects, the similarity label indicates that the label image and the sample image are not similar, that is, the electronic handwritten signature in the label image is not similar to the electronic handwritten signature in the sample image, for example, represented by "0".

[0097] Step 502: Perform the following processing based on the model to be trained: extract the first sample features of the sample image and the second sample features of the label image, perform signature recognition based on the first sample features to obtain signature recognition information, and determine similarity prediction information based on the first sample features and the second sample features.

[0098] In this embodiment, the training device includes a model to be trained, or the training device can obtain the model to be trained from other devices (e.g., pre-training devices, storage devices, etc.). The model to be trained is a neural network model for verifying electronic handwritten signatures. Therefore, after obtaining training samples, the training device uses the training samples to train the model to be trained. Here, the training device includes three processes during model training: feature extraction, signature recognition, and similarity prediction (also known as similarity calculation). Feature extraction includes feature extraction from sample images and label images. The first sample feature refers to the feature of the sample image extracted through feature extraction, and the second sample feature refers to the feature of the label image extracted through feature extraction. Signature recognition refers to recognizing the electronic handwritten signature of the sample image, achieved through the first sample feature. Alternatively, it can refer to recognizing the electronic handwritten signature of the signature image, achieved through the second sample feature (in this case, the training samples also include the annotation information of the electronic handwritten signature of the signature image). Similarity prediction is used to predict whether the sample image and the signature image are similar, determined by comparing the first sample feature and the second sample feature.

[0099] It should be noted that the model to be trained can be an originally constructed neural network model for verifying electronic handwritten signatures, a neural network model for verifying electronic handwritten signatures that has been trained at least once, or a pre-trained neural network model for verifying electronic handwritten signatures, etc., and this application does not limit this. Signature recognition information refers to the prediction result of the electronic handwritten signature of the sample image, including at least one of the predicted electronic handwritten signature's position and content; similarity prediction information refers to the prediction result of whether the label image and the sample image are similar, which can be that the label image and the sample image are similar, or that the label image and the sample image are dissimilar.

[0100] Step 503: Combine the differences between signature recognition information and signature labels, as well as the differences between similarity prediction information and similar labels, to train the model to be trained and obtain the signature verification model.

[0101] In this embodiment, after obtaining signature recognition information, the training device compares the signature recognition information with the signature label. Based on the differences between the signature recognition information and the signature label, and the differences between the similarity prediction information and the similar labels, backpropagation is performed in the model to be trained to adjust the model parameters. A signature verification model is then obtained based on the trained model. Here, the difference between the signature recognition information and the signature label represents the loss of the text recognition task, the difference between the similarity prediction information and the similar labels represents the loss of the similarity calculation task, and the signature verification model is used to verify the electronic handwritten signature on the signature image to be verified.

[0102] It should be noted that the training device obtains the signature verification model based on the trained model to be trained. This can be achieved by directly determining the trained model to be trained as the signature verification model, or by continuing to train the trained model to be trained using the training process described in steps 502 and 503 to obtain the signature verification model, etc. The embodiments of this application do not limit this. In addition, the signature verification model refers to the latest neural network model obtained for verifying electronic handwritten signatures.

[0103] It should also be noted that the training of the model to be trained can be carried out iteratively. The training ends when the training termination condition is met, and the signature verification model is obtained. Here, the training termination condition can be reaching the accuracy index threshold, the number of training sessions threshold, the training duration threshold, or a combination of the above, etc. This application does not limit this.

[0104] Understandably, for a trainable model that can be used for both signature recognition and similarity prediction, by acquiring training samples and using these samples to train the model for both signature recognition and similarity prediction, the overall training of both tasks—signature recognition and image similarity prediction—is achieved within a single model. This reduces the complexity of model training and improves the overall verification performance of both tasks. Therefore, it can improve the efficiency of model training and the accuracy of model verification.

[0105] See Figure 6 , Figure 6 This is a flowchart illustrating the training method of the signature verification model provided in the embodiments of this application. Figure 2 ;like Figure 6 As shown in the embodiment of this application, step 503 can be implemented through steps 5031 to 5033; that is, the training device combines the difference between signature recognition information and signature tags, as well as the difference between similarity prediction information and similar tags, to train the model to be trained and obtain the signature verification model, including steps 5031 to 5033. Each step is described below.

[0106] Step 5031: The difference between the signature recognition information and the signature label is fused with the first difference weight to obtain the first target difference.

[0107] In this embodiment, the training device is equipped with a first difference weight, or the training device can obtain the first difference weight from other devices or received instructions; the first difference weight represents the proportion of the difference between the signature recognition information and the signature label in the loss used to train the model to be trained; it can be specified, or it can be automatically adjusted during training, etc., and this embodiment does not limit it in this way. Here, the training device fuses the difference between the signature recognition information and the signature label with the first difference weight, and the result obtained is the first target difference; wherein, the first target difference is the signature recognition loss used when training the model to be trained.

[0108] Step 5032: The difference between the similarity prediction information and the similar labels is fused with the second difference weight to obtain the second target difference.

[0109] In this embodiment, the training device is equipped with a second difference weight, or the training device can obtain the second difference weight from other devices or received instructions; the second difference weight represents the proportion of the difference between the similarity prediction information and the similar labels in the loss used to train the model to be trained; it can be specified, or it can be automatically adjusted during training, etc., and this embodiment does not limit this. Here, the training device fuses the difference between the similarity prediction information and the similar labels with the second difference weight, and the result obtained is the second target difference; wherein, the second target difference is the similarity calculation loss used when training the model to be trained.

[0110] Step 5033: Combine the differences between the first objective and the second objective to train the model to be trained, and obtain the signature verification model.

[0111] In this embodiment, after obtaining the first target difference and the second target difference, the training device combines the two to train the model to be trained until a signature verification model is obtained. The combination of the first target difference and the second target difference can be addition, multiplication, etc., and this embodiment does not limit the specific combination.

[0112] Understandably, the training device adjusts the loss of the signature recognition part and the loss of the similarity calculation part by using the first difference weight and the second difference weight. It can train the model based on the impact of the two tasks of signature recognition and similarity calculation on the model, so that the trained signature verification model can perform signature verification in a targeted manner.

[0113] In this embodiment, the training method for the signature verification model further includes a process of adjusting the first difference weight and the second difference weight by the training device: During the training of the model to be trained, the training device acquires test samples; and verifies the electronic handwritten signature of the test samples based on the currently trained model to be trained, obtaining the signature accuracy and similarity accuracy; when the signature accuracy is greater than the similarity accuracy, a first adjustment is made to at least one of the first difference weight and the second difference weight to obtain a first target difference weight that is less than the second target difference weight; and when the signature accuracy is less than the similarity accuracy, a second adjustment is made to at least one of the first difference weight and the second difference weight to obtain a first target difference weight that is greater than the second target difference weight; finally, based on the first target difference weight and the second target difference weight, a target loss for training the currently trained model to be trained is obtained.

[0114] It should be noted that electronic handwritten signature verification includes signature recognition and similarity calculation. Signature accuracy refers to the accuracy of matching the recognized signature text with the labeled signature, while similarity accuracy refers to the accuracy of the similarity result calculated in the similarity calculation being consistent with the labeled similarity. Here, when the signature accuracy is greater than the similarity accuracy, it indicates that the accuracy of the similarity calculation needs to be improved. Therefore, at least one of the first difference weight and the second difference weight is adjusted so that the adjusted first target difference weight corresponding to the first difference weight is less than the second target difference weight corresponding to the second difference weight. Conversely, when the signature accuracy is less than the similarity accuracy, it indicates that the accuracy of signature recognition needs to be improved. Therefore, at least one of the first difference weight and the second difference weight is adjusted so that the adjusted first target difference weight corresponding to the first difference weight is greater than the second target difference weight corresponding to the second difference weight. Furthermore, when the signature accuracy equals the similarity accuracy, the training device may not adjust the first and second difference weights, or it may adjust them based on the signature accuracy threshold and the similarity accuracy threshold.

[0115] It is understandable that by dynamically adjusting the first and second difference weights during the training process of the model to be trained, the model can be dynamically trained in a targeted manner based on the current signature accuracy and similarity accuracy, thereby improving the accuracy of model training.

[0116] In this embodiment, the model to be trained includes a feature extraction module, which includes a first sub-feature extraction module and a second sub-feature extraction module. The first and second sub-feature extraction modules share model parameters. In step 502, the training device extracts the first sample features of the sample image and the second sample features of the label image, which includes: the training device extracts the first sample features of the sample image through the first sub-feature extraction module and the second sample features of the label image through the second sub-feature extraction module. That is, the first and second sub-feature extraction modules are identical modules used for feature extraction.

[0117] It is understandable that since the first and second sub-feature extraction modules share model parameters, the training efficiency of the feature extraction module can be improved, which in turn can improve the training efficiency of the signature verification model.

[0118] In the embodiments of this application, the signature label includes at least one of a signature location label and a signature content label; wherein, the signature location label indicates the location of the marked signature, and the signature content label indicates the marked signature content.

[0119] In this embodiment of the application, when the signature label includes a signature location label and a signature content label, in step 502, the training device performs signature recognition based on the first sample features to obtain signature recognition information, including: the training device performs signature location detection based on the first sample features to obtain location prediction information; and performs signature content recognition based on the first sample features to obtain content prediction information; finally, the location prediction information and the content prediction information are determined as signature recognition information.

[0120] It should be noted that signature recognition includes signature location detection and signature content recognition. Signature location detection refers to the process of detecting the position of the electronic handwritten signature in an image, while signature content recognition refers to the process of recognizing the content of the electronic handwritten signature. The location prediction information is the result obtained when the model to be trained performs signature location detection on the first sample features; it is a predicted signature location. The content prediction information is the result obtained when the model to be trained performs signature content recognition on the first sample features; it is a predicted signature content.

[0121] Accordingly, in the embodiments of this application, the training method of the signature verification model also includes the process of the difference between signature recognition information and signature label: the training device combines the difference between location prediction information and signature location label, and the difference between content prediction information and signature content label, into the difference between signature recognition information and signature label.

[0122] In this embodiment of the application, before the training device combines the difference between the location prediction information and the signature location tag, and the difference between the content prediction information and the signature content tag, into the difference between the signature recognition information and the signature tag, the training method based on the signature verification model further includes: the training device obtaining the differences of each position parameter between the location prediction information and the signature location tag, and combining the differences of each position parameter into the difference between the signature location prediction information and the signature location tag.

[0123] It should be noted that the differences in each position parameter correspond one-to-one with each position parameter, which is used to represent the signature position; for example, the two-dimensional coordinates, length, width, etc. of the specified position point (center point, to show the upper left, lower left, upper right, and lower right corners of the viewpoint) of the electronic handwritten signature.

[0124] It is understandable that when the signature label includes both the signature position label and the signature content label, the loss of the signature recognition task is determined by both the signature position and the signature content. This allows the loss of the signature recognition task to be obtained from multiple dimensions, which can improve the accuracy of model training.

[0125] In this embodiment, the signature content prediction information includes multiple prediction scores corresponding to multiple content categories, where each content category represents a type of content. Therefore, the signature content tag is also a content category among the multiple content categories. Here, before the training device combines the differences between the signature position prediction information and the signature position tag, and the differences between the signature content prediction information and the signature content tag, into the difference between the signature prediction information and the signature tag, the training method for the signature verification model also includes a process for obtaining the difference between the signature content prediction information and the signature content tag: the training device normalizes the prediction score corresponding to the signature content tag based on the multiple prediction scores, thus obtaining the normalized result of the prediction score corresponding to the signature content tag, which is the difference between the signature content prediction information and the signature content tag.

[0126] In this embodiment, the sample image includes at least one of a positive sample image and a negative sample image; wherein, the positive sample image and the label image originate from the same signature object, and the difference between the positive sample image and the label image may be a difference in signature size, a difference in signature language, a difference in signature writing style (e.g., simplification, font style, etc.), or a combination of the above, etc., and this embodiment does not limit this. The negative sample image and the label image originate from different signature objects.

[0127] In this embodiment, when the sample image includes a positive sample image and a negative sample image, the first sample feature includes a first sub-sample feature of the positive sample image and a second sub-sample feature of the negative sample image. Therefore, in step 502, the training device determines the similarity prediction information based on the first sample feature and the second sample feature, including: the training device obtains a first feature distance between the first sub-sample feature and the second sample feature; and obtains a second feature distance between the second sub-sample feature and the second sample feature, and determines the first feature distance and the second feature distance as the similarity prediction information. At this time, the process by which the training device obtains the difference between the similarity prediction information and the similar label is as follows: the training device obtains the feature distance difference between the first feature distance and the second feature distance based on the similar label; and determines the difference between the similarity prediction information and the similar label based on the comparison result of the feature distance difference and the feature distance threshold.

[0128] It should be noted that the first feature distance and the second feature distance are similarity prediction information. Here, the training device obtains the feature distance difference between the first feature distance and the second feature distance based on the similar labels, including: the training device determines the first feature distance as the subtrahend and the second feature distance as the minuend based on the similar labels, so as to calculate the feature distance difference between the first feature distance and the second feature distance through subtraction. Furthermore, the feature distance threshold is a specified feature distance, such as 0, or a zero matrix, etc. In addition, when the feature distance difference is less than the feature distance threshold, the training device determines the feature distance threshold as the difference between the similarity prediction information and the similar labels; while when the feature distance difference is greater than or equal to the feature distance threshold, the training device determines the feature distance difference as the difference between the similarity prediction information and the similar labels.

[0129] It should also be noted that when the sample image includes both positive and negative sample images, the signature recognition information includes the positive signature recognition information of the positive sample image and the negative signature recognition information of the negative sample image, and the signature label includes the positive signature label of the positive sample image and the negative signature label of the negative sample image; the difference between the signature recognition information and the signature label includes a weighted combination of the following two items: the difference between the positive signature recognition information and the positive signature label, and the difference between the negative signature recognition information and the negative signature label.

[0130] In this embodiment, when the sample image is a positive sample image, the training device can obtain the difference between the similarity prediction information and the similar labels through the following steps: obtaining a first similarity difference between the similarity prediction information and the similar labels; and determining the first similarity difference as the difference between the similarity prediction information and the similar labels. When the sample image is a negative sample image, the training device can obtain the difference between the similarity prediction information and the similar labels through the following steps: obtaining a second similarity difference between the similarity prediction information and the similar labels; and determining the second similarity difference as the difference between the similarity prediction information and the similar labels.

[0131] In this embodiment of the application, the training device may also perform the following processing using the model to be trained: perform signature recognition based on the features of the second sample to obtain the signature recognition result of the label image; thereby, the training device combines the difference between the signature recognition information and the signature label, as well as the difference between the similarity prediction information and the similar label, to train the model to be trained, including: combining the difference between the signature recognition information and the signature label, the difference between the signature recognition result of the label image and the annotation information of the electronic handwritten signature of the label image, as well as the difference between the similarity prediction information and the similar label, to train the model to be trained.

[0132] In this embodiment of the application, after the training device obtains the signature verification model in step 503, the training method of the signature verification model further includes: the training device acquiring new training samples, wherein the new training samples include a new label image, a new sample image, a new signature label of the new sample image, and a new similar label between the new label image and the new sample image, wherein the new label image and the new sample image are both electronic handwritten signature images; and performing the following processing based on the signature verification model: extracting a first new sample feature of the new sample image and a second new sample feature of the new label image, performing signature recognition on the first new sample feature to obtain new signature recognition information, determining new similarity prediction information based on the first new sample feature and the second new sample feature; and optimizing the signature verification model by combining the difference between the new signature recognition information and the new signature label, and the difference between the new similarity prediction information and the new similar label, to obtain a target signature verification model, wherein the target signature verification model is used to verify the electronic handwritten signature of a new signature image to be verified or a signature image to be verified.

[0133] It should be noted that the process by which the training device optimizes the signature verification model based on new training samples to obtain the target signature verification model is similar to the process by which the training device trains the model to be trained based on training samples to obtain the signature verification model. Therefore, the embodiments of this application will not be described again here.

[0134] Understandably, after obtaining the signature verification model, the training device collects new training samples to optimize the signature verification model, which can improve the generalization ability of the signature verification model.

[0135] The following describes an exemplary application of the embodiments of this application in a practical application scenario. This exemplary application describes the process of training a signature verification model for verifying electronic handwritten signatures in a security authentication scenario, and using the signature verification model to perform electronic handwritten signature verification.

[0136] See Figure 7 , Figure 7 This is an exemplary flowchart of an electronic handwritten signature verification process provided in an embodiment of this application; as follows:Figure 7 As shown, this exemplary electronic handwritten signature verification process includes a model training phase 7-1, an initialization phase 7-2, and a verification phase 7-3.

[0137] In the model training phase 7-1, model pre-training is performed 7-11, and dataset collection and labeling are conducted 7-12. Then, the pre-trained model is trained using the collected and labeled dataset 7-13, resulting in the final signature verification model 7-14. Specifically, during the dataset collection and labeling process 7-12, multiple electronic handwritten signature images (referred to as M electronic handwritten signature images) are collected from each of multiple individuals, and each handwritten signature image is labeled to obtain the text location. (Referred to as signature location tag) and text content (Referred to as signature content tag), text location Including the x-axis y-axis ,width and height (These are called the positional parameters).

[0138] In the initialization phase 7-2, the user's input name information 7-21 (called the initial signature text) and handwritten signature image 7-22 (called the initial signature image) are first obtained. Then, the signature verification model 7-14 is used to perform text recognition on the handwritten signature image 7-22 to identify the name information 7-23 (called the signature text recognition result). Next, it is determined whether the name information 7-23 matches the name information 7-21. If they match, the initialization of the name information 7-21 and the handwritten signature image 7-22 is completed, and the name information 7-21 and the handwritten signature image 7-22 are stored in the information corresponding to the user's account. If they do not match, the initialization fails.

[0139] In the verification phase 7-3, the handwritten signature image 7-31 to be verified, input by the user, is first obtained. Then, the handwritten signature image 7-31 and the handwritten signature image 7-22 are input into the signature verification model 7-14. First, text recognition is performed on the handwritten signature image 7-31 to identify the name information 7-32 (called the target signature text). Second, the similarity between the handwritten signature image 7-31 and the handwritten signature image 7-22 is obtained. Next, the matching result between the name information 7-32 and the name information 7-21 is determined. Finally, based on the similarity and matching result, the handwritten signature verification result is obtained. Here, when the similarity is greater than the similarity threshold and the name information 7-32 and the name information 7-21 match, the handwritten signature verification result indicates that the handwritten signature verification is successful, and the corresponding processing can be completed (such as account login, password modification, binding information modification, status unblocking, etc.).

[0140] The following section provides a detailed explanation of the model training phase 7-1.

[0141] See Figure 8 , Figure 8 This is an exemplary model structure diagram provided in an embodiment of this application; as shown... Figure 8 As shown, it includes a convolutional neural network module 8-1, a text recognition module 8-2, and an image similarity calculation module 8-3. Furthermore, the convolutional neural network module 8-1, the text recognition module 8-2, and the image similarity calculation module 8-3 can be obtained through model pre-training 7-11. The convolutional neural network module 8-1 includes a feature extraction module 8-11, a feature extraction module 8-12 (together with the feature extraction module 8-11, referred to as the first sub-feature extraction module), and a feature extraction module 8-13 (referred to as the second sub-feature extraction module). The feature extraction modules 8-11, 8-12, and 8-13 share parameters. The feature extraction module 8-11 is used to extract features from the target image 8-41(T) to obtain image features 8-51(t). The feature extraction module 8-12 is used to extract features from the positive example image 8-42 (P, which is randomly selected from images of the same source as the target image) to obtain image features 8-52(p). The feature extraction module 8-13 is used to extract features from the negative example image 8-43 (N, which is randomly selected from images of different sources as the target image) to obtain image features 8-53(n).

[0142] The text recognition module 8-2 includes text recognition submodules 8-21, 8-22, and 8-23, which share parameters. Text recognition submodules 8-21, 8-22, and 8-23 perform text location detection and text content recognition on image features 8-51, yielding a location loss 8-611 (referred to as the difference between location prediction information and signature location label) and a text content loss 8-612 (referred to as the difference between content prediction information and signature content label), respectively. Text recognition submodule 8-22 performs text location detection and text content recognition on image features 8-52, yielding a location loss 8-621 and a text content loss 8-622, respectively. Text recognition submodule 8-23 performs text location detection and text content recognition on image features 8-53, yielding a location loss 8-631 and a text content loss 8-632, respectively. Here, the location loss... It can be obtained through formula (1) and formula (2), which are shown below.

[0143] (1);

[0144] (2);

[0145] Among them, the text position detected is the text position. Including the x-axis y-axis ,width and height (These are called the positional parameters).

[0146] Text content loss It can be obtained through formulas (3) and (4), which are shown below.

[0147] (3);

[0148] (4);

[0149] in, For the sample Predicted as category The probability, For the text recognition submodule, samples exist The output of the class, where C is the number of classes and samples. Image features, category This is the actual text content.

[0150] The image similarity calculation module 8-3 includes fully connected layers 8-31 and 8-32, which share parameters. Fully connected layer 8-31 processes image features 8-51 and 8-52, while fully connected layer 8-32 processes image features 8-51 and 8-53. Based on the two processing results, the image similarity loss 8-64 (referred to as the difference between similarity prediction information and similar labels) is obtained.

[0151] Image similarity loss It can be obtained through formula (5), which is shown below.

[0152] (5);

[0153] in, Indicates the distance to Jaccard. Used to obtain the maximum value.

[0154] Here, model loss 8-6 is obtained by combining position loss 8-611 and text content loss 8-612, position loss 8-611 and text content loss 8-612, position loss 8-631 and text content loss 8-63, and image similarity loss 8-64. Model loss 8-6 is then used to train convolutional neural network module 8-1, text recognition module 8-2, and image similarity calculation module 8-3 to obtain the signature verification model. (Model loss) It can be obtained through formula (6), which is shown below.

[0155] (6);

[0156] Where m is the number of samples, This is used to control the degree to which text recognition and image similarity affect the overall result.

[0157] The structures of the convolutional neural network module 8-1 and the text recognition module 8-2 will be described below.

[0158] For example, see Figure 9 , Figure 9 This is a schematic diagram of the structure of an exemplary feature extraction model provided in an embodiment of this application; as shown... Figure 9 As shown, the feature extraction model structure 9-1 is the structure of any feature extraction model in the convolutional neural network module 8-1, including convolutional modules 9-11 to 9-13, as well as upsampling modules 9-14 and 9-15; wherein, each convolutional module in convolutional modules 9-11 to 9-13 includes a convolutional layer (Conv), a pooling layer, and an activation function layer (ReLU). Here, each image 9-21 in the target image 8-41, positive example image 8-42, and negative example image 8-43 is processed by convolution module 9-11 to obtain feature 9-22; feature 9-22 is processed by convolution module 9-12 to obtain feature 9-23; feature 9-23 is processed by convolution module 9-13 to obtain feature 9-24; feature 9-24 is processed by upsampling module 9-14, and the obtained feature is combined with feature 9-23 to obtain feature 9-25; feature 9-25 is processed by upsampling module 9-15, and the obtained feature is combined with feature 9-22 to obtain image feature 9-26; and image feature 9-26 is any one of image features 8-51 to image features 8-53.

[0159] For example, see Figure 10 , Figure 10 This is a schematic diagram of the structure of an exemplary text recognition module provided in an embodiment of this application; as shown... Figure 10As shown, the text recognition module 8-2 includes convolution module 10-11, convolution module 10-12, reshaping module 10-13 (Reshape), normalization module 10-14 (Softmax), reshaping module 10-15, generation module 10-16 (Proposal), region of interest pooling module 10-17 (ROI Pooling), fully connected module 10-18, and fully connected module 10-19. Specifically, convolutional modules 10-11 and 10-12 are used to process image features 9-26, and each convolutional module in convolutional modules 10-11 and 10-12 includes a convolutional layer and a geometric function layer; the input of generation module 10-16 includes the output of reshaping module 10-15, the output of convolutional module 10-12, and the size information 10-2 of image 9-21; the input of region of interest pooling module includes the output of generation module 10-16 and image features 9-26; fully connected module 10-18 is used to process the output of region of interest pooling module 10-17 regarding text position to obtain position loss; fully connected module 10-19 is used to process the output of region of interest pooling module 10-17 regarding text content to obtain text content loss.

[0160] The following explains the relationship between the terminal, storage, and server.

[0161] based on Figure 7 See Figure 11 , Figure 11 This is an exemplary device interaction diagram provided in an embodiment of this application; as shown below. Figure 11 As shown, terminal 11-1 can obtain the user-input name information 7-21 and handwritten signature image 7-22 during initialization. Then, terminal 11-1 sends the name information 7-21 and handwritten signature image 7-22 to server 11-2. At this time, server 11-2 retrieves signature verification model 7-14 from memory 11-3 and uses signature verification model 7-14 to determine whether the name information 7-23 identified from handwritten signature image 7-22 matches the name information 7-21. If a match is determined, the name information 7-21 and handwritten signature image 7-22 are stored in the target account information in memory 11-3. Terminal 11-1 can also receive user-inputted account information 11-4 and handwritten signature image 7-31 to be verified. Then, terminal 11-1 sends the account information 11-4 and handwritten signature image 7-31 to be verified to server 11-2. At this time, server 11-2 retrieves name information 7-21 and handwritten signature image 7-22 from memory 11-3 based on account information 11-4, and uses signature verification model 7-14 to process the handwritten signature image 7-31, name information 7-21 and handwritten signature image 7-22 to complete signature recognition and similarity calculation.

[0162] The following describes the application scenarios of electronic handwritten signature verification provided in the embodiments of this application.

[0163] See Figure 12 , Figure 12 This is a schematic diagram illustrating an exemplary post-processing scenario provided in an embodiment of this application; as shown... Figure 12 As shown, in the login interface 12-1, application login can be achieved through account (e.g., username, mobile phone number, email) and electronic handwritten signature verification (as shown in control 12-11), or through password login verification (as shown in control 12-12). Here, when control 12-11 is triggered, the handwritten signature input panel interface will be displayed. Responding to the signature input operation on the handwritten signature input panel interface, the handwritten signature image 7-31 to be verified is obtained. When changing the password in the password modification interface 12-2, identity verification can be achieved through account (e.g., username, mobile phone number, email) and password verification (as shown in control 12-21), or through document-assisted verification (as shown in control 12-22), or through electronic handwritten signature verification (as shown in control 12-23). When modifying binding information in the binding information modification interface 12-3, identity verification can be achieved through account (e.g., username, mobile phone number, email) and password verification (as shown in control 12-31), or through document-assisted verification (as shown in control 12-32), or through electronic handwritten signature verification (as shown in control 12-33). When unblocking status in the status unblocking interface 12-4, identity verification can be achieved through electronic handwritten signature verification (as shown in control 12-41).

[0164] It's important to note that password verification is the primary user authentication method in login scenarios, verifying the password corresponding to the account to complete user authentication. SMS verification involves cooperation with telecom operators; users receive or upload a designated SMS verification code via their security phone, and the network company verifies the code's correctness to complete user authentication. Document-assisted verification is essentially a combination of facial recognition and ID card verification. For facial recognition, the network company collects user facial data and compares the facial images for consistency to complete authentication. For ID card verification, the network company collects the user's name and ID card information, confirms the information through government-open interfaces, and completes authentication. Additionally, electronic handwritten signature verification offers high security, requires no additional cost, and does not require the collection of sensitive user information.

[0165] Understandably, the tasks of text recognition and image similarity calculation can be performed in the same neural network model through hyperparameters. Adjusting the impact of the two tasks on the signature verification model enables end-to-end training and improves the efficiency of electronic handwritten signature verification. The feature extraction modules for text recognition and image similarity calculation share parameters, improving overall performance through multi-task learning. Furthermore, this application's embodiments improve the efficiency of electronic handwritten signature verification, and verification via electronic handwritten signatures enhances verification security.

[0166] The following description continues to illustrate the exemplary structure of the training device 255 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 3 As shown, the software modules stored in the training device 255 in the first memory 250 may include:

[0167] The sample acquisition module 2551 is used to acquire training samples, wherein the training samples include label images, sample images, signature labels of the sample images, and similar labels between the label images and the sample images, wherein the label images and the sample images are both electronic handwritten signature images;

[0168] The model verification module 2552 is used to perform the following processing based on the model to be trained: extracting the first sample features of the sample image and the second sample features of the label image, performing signature recognition based on the first sample features to obtain signature recognition information, and determining similarity prediction information based on the first sample features and the second sample features;

[0169] The model training module 2553 trains the model to be trained by combining the difference between the signature recognition information and the signature label, and the difference between the similarity prediction information and the similarity label, to obtain the signature verification model, wherein the signature verification model is used to verify the electronic handwritten signature of the signature image to be verified.

[0170] In this embodiment of the application, the model training module 2553 is further configured to fuse the difference between the signature recognition information and the signature label with a first difference weight to obtain a first target difference; fuse the difference between the similarity prediction information and the similar label with a second difference weight to obtain a second target difference; and combine the first target difference and the second target difference to train the model to be trained to obtain the signature verification model.

[0171] In this embodiment, the model training module 2553 is further configured to: acquire test samples during the training of the model to be trained; verify the electronic handwritten signature of the test samples based on the currently trained model to be trained, and obtain signature accuracy and similarity accuracy; when the signature accuracy is greater than the similarity accuracy, perform a first adjustment on at least one of the first difference weight and the second difference weight to obtain a first target difference weight that is less than the second target difference weight; when the signature accuracy is less than the similarity accuracy, perform a second adjustment on at least one of the first difference weight and the second difference weight to obtain a first target difference weight that is greater than the second target difference weight; and obtain a target loss for training the currently trained model to be trained based on the first target difference weight and the second target difference weight.

[0172] In this embodiment of the application, the model to be trained includes a feature extraction module, which includes a first sub-feature extraction module and a second sub-feature extraction module. The first sub-feature extraction module and the second sub-feature extraction module share model parameters. The model training module 2553 is further configured to extract the first sample features of the sample image through the first sub-feature extraction module and the second sample features of the label image through the second sub-feature extraction module.

[0173] In this embodiment of the application, the signature tag includes at least one of a signature location tag and a signature content tag. When the signature tag includes the signature location tag and the signature content tag, the model training module 2553 is further configured to perform signature location detection based on the first sample features to obtain location prediction information; perform signature content recognition based on the first sample features to obtain content prediction information, wherein the signature recognition includes the signature location detection and the signature content recognition; and determine the location prediction information and the content prediction information as the signature recognition information.

[0174] In this embodiment of the application, the model training module 2553 is further configured to combine the difference between the location prediction information and the signature location tag, and the difference between the content prediction information and the signature content tag, into the difference between the signature recognition information and the signature tag.

[0175] In this embodiment of the application, the model training module 2553 is further configured to include at least one of positive sample images and negative sample images, wherein the positive sample images and the label images originate from the same signature object, and the negative sample images and the label images originate from different signature objects.

[0176] In this embodiment of the application, when the sample image includes a positive sample image and a negative sample image, the first sample feature includes a first sub-sample feature of the positive sample image and a second sub-sample feature of the negative sample image. The model training module 2553 is further configured to obtain a first feature distance between the first sub-sample feature and the second sample feature; obtain a second feature distance between the second sub-sample feature and the second sample feature; and determine the first feature distance and the second feature distance as the similarity prediction information.

[0177] In this embodiment of the application, the model training module 2553 is further configured to obtain the feature distance difference between the first feature distance and the second feature distance based on the similarity label; and determine the difference between the similarity prediction information and the similarity label based on the comparison result of the feature distance difference and the feature distance threshold.

[0178] In this embodiment, the model training module 2553 is further configured to acquire new training samples, wherein the new training samples include a new label image, a new sample image, a new signature label of the new sample image, and a new similar label between the new label image and the new sample image, wherein the new label image and the new sample image are both electronic handwritten signature images; and the following processing is performed based on the signature verification model: extracting a first new sample feature of the new sample image and a second new sample feature of the new label image, performing signature recognition on the first new sample feature to obtain new signature recognition information, determining new similarity prediction information based on the first new sample feature and the second new sample feature; and optimizing the signature verification model by combining the difference between the new signature recognition information and the new signature label, and the difference between the new similarity prediction information and the new similar label, to obtain a target signature verification model, wherein the target signature verification model is used to verify the electronic handwritten signature of a new signature image to be verified.

[0179] The following description continues to illustrate the exemplary structure of the data processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software modules stored in the data processing device 455 of the second memory 450 may include:

[0180] The signature verification module 4551 is used to verify the electronic handwritten signature of the signature image to be verified and the standard signature image based on the signature verification model, and to obtain the target signature text of the signature image to be verified and the target similarity between the signature image to be verified and the standard signature image. The signature verification model is trained by the training method of the signature verification model described above.

[0181] The result processing module 4552 is used to combine the matching result between the target signature text and the standard signature text, and the comparison result between the target similarity and the similarity threshold, to determine the verification result of the signature image to be verified.

[0182] In this embodiment of the application, the data processing device 455 further includes an initialization module 4553, which is used to acquire an initialization signature image and an initialization signature text; perform signature recognition on the initialization signature image based on the signature verification model to obtain a signature text recognition result; when the signature text recognition result matches the standard signature text, the initialization signature image is determined as the standard signature image, and the initialization signature text is determined as the standard signature text.

[0183] In this embodiment of the application, the result processing module 4552 is further configured to display a signature interface in response to a trigger operation on the verification control, wherein the verification control is configured to trigger the acquisition of the signature image to be verified; and to obtain the signature image to be verified in response to a signature operation on the signature interface.

[0184] In this embodiment of the application, the result processing module 4552 is further configured to perform target processing when the verification result indicates that the target signature text and the standard signature text match and the target similarity is greater than the similarity threshold. The target processing includes at least one of account login, information update and account status unblocking.

[0185] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the signature verification-based data processing method described above in this application.

[0186] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 5 The method shown.

[0187] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0188] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0189] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0190] As an example, computer-executable instructions can be deployed to execute on a single electronic device (in which case, this single electronic device is both a training device and a data processing device), or to execute on multiple electronic devices located at one location (in which case, the multiple electronic devices located at one location are both a training device and a data processing device), or to execute on multiple electronic devices distributed across multiple locations and interconnected via a communication network (in which case, the multiple electronic devices distributed across multiple locations and interconnected via a communication network are both a training device and a data processing device).

[0191] It is understood that in the embodiments of this application, data such as images are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0192] In summary, in this embodiment, for a trainable model capable of both signature recognition and similarity prediction, training samples are acquired and used to train the model for both signature recognition and similarity prediction. This achieves integrated training of both tasks within a single model, reducing training complexity and thus improving training efficiency. Furthermore, combining signature recognition and image similarity prediction for integrated model training allows for flexible adjustment of the relative importance of each task within the signature verification model, thereby enhancing prediction accuracy.

[0193] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a signature verification model, characterized in that, The method includes: Obtain training samples, wherein the training samples include label images, sample images, signature labels of the sample images, and similar labels between the label images and the sample images, wherein the label images and the sample images are both electronic handwritten signature images; The following processing is performed based on the model to be trained: extracting the first sample features of the sample image and the second sample features of the label image, performing signature recognition based on the first sample features to obtain signature recognition information, and determining similarity prediction information based on the first sample features and the second sample features; The difference between the signature recognition information and the signature tag is fused with a first difference weight to obtain a first target difference; The difference between the similarity prediction information and the similar labels is fused with a second difference weight to obtain the second target difference; By combining the first target difference and the second target difference, the model to be trained is trained to obtain the signature verification model, wherein the signature verification model is used to verify the electronic handwritten signature of the signature image to be verified.

2. The method according to claim 1, characterized in that, The method further includes: During the training of the model to be trained, test samples are obtained; The electronic handwritten signature of the test sample is verified based on the currently trained model to obtain the signature accuracy and similarity accuracy. When the signature accuracy is greater than the similarity accuracy, at least one of the first difference weight and the second difference weight is adjusted to obtain a first target difference weight that is less than the second target difference weight. When the signature accuracy is less than the similarity accuracy, at least one of the first difference weight and the second difference weight is adjusted to obtain a first target difference weight that is greater than the second target difference weight. Based on the first target difference weight and the second target difference weight, the target loss for training the model to be trained is obtained.

3. The method according to any one of claims 1 to 2, characterized in that, The model to be trained includes a feature extraction module, which includes a first sub-feature extraction module and a second sub-feature extraction module. The first sub-feature extraction module and the second sub-feature extraction module share model parameters. The extraction of the first sample feature of the sample image and the second sample feature of the label image includes: The first sample feature of the sample image is extracted by the first sub-feature extraction module, and the second sample feature of the label image is extracted by the second sub-feature extraction module.

4. The method according to any one of claims 1 to 2, characterized in that, The signature label includes at least one of a signature location label and a signature content label. When the signature label includes both the signature location label and the signature content label, the signature recognition based on the first sample features to obtain signature recognition information includes: Based on the features of the first sample, signature location detection is performed to obtain location prediction information; Based on the features of the first sample, signature content recognition is performed to obtain content prediction information, wherein the signature recognition includes the signature position detection and the signature content recognition; The location prediction information and the content prediction information are determined as the signature identification information; The method further includes: The difference between the location prediction information and the signature location tag, and the difference between the content prediction information and the signature content tag, are combined to form the difference between the signature identification information and the signature tag.

5. The method according to claim 1, characterized in that, The sample image includes at least one of a positive sample image and a negative sample image, wherein the positive sample image and the label image originate from the same signature object, and the negative sample image and the label image originate from different signature objects.

6. The method according to any one of claims 1 to 2, 5, characterized in that, When the sample image includes positive sample images and negative sample images, the first sample feature includes a first sub-sample feature of the positive sample image and a second sub-sample feature of the negative sample image. The step of determining similarity prediction information based on the first sample feature and the second sample feature includes: Obtain the first feature distance between the features of the first subsample and the features of the second sample; Obtain the second feature distance between the second subsample feature and the second sample feature; The first feature distance and the second feature distance are determined as the similarity prediction information; The method further includes: Based on the similar labels, the feature distance difference between the first feature distance and the second feature distance is obtained; Based on the comparison result of the feature distance difference and the feature distance threshold, the difference between the similarity prediction information and the similar labels is determined.

7. The method according to any one of claims 1 to 2, 5, characterized in that, After obtaining the signature verification model, the method further includes: Obtain new training samples, wherein the new training samples include new label images, new sample images, new signature labels of the new sample images, and new similar labels between the new label images and the new sample images, wherein the new label images and the new sample images are both electronic handwritten signature images; Based on the signature verification model, the following processing is performed: extracting the first new sample feature of the new sample image and the second new sample feature of the new label image; performing signature recognition on the first new sample feature to obtain new signature recognition information; and determining new similarity prediction information based on the first new sample feature and the second new sample feature. By combining the differences between the new signature recognition information and the new signature label, and the differences between the new similarity prediction information and the new similarity label, the signature verification model is optimized to obtain a target signature verification model, wherein the target signature verification model is used to verify the electronic handwritten signature of a new signature image to be verified.

8. A data processing method based on signature verification, characterized in that, The method includes: The signature verification model is used to verify the electronic handwritten signature of the signature image to be verified and the standard signature image, so as to obtain the target signature text of the signature image to be verified and the target similarity between the signature image to be verified and the standard signature image. The signature verification model is trained by the training method described in any one of claims 1 to 7. The verification result of the signature image to be verified is determined by combining the matching results between the target signature text and the standard signature text, and the comparison results between the target similarity and the similarity threshold.

9. The method according to claim 8, characterized in that, Before verifying the electronic handwritten signature of the image to be verified and the standard signature image based on the signature verification model, the method further includes: Obtain the initial signature image and initial signature text; Based on the signature verification model, the initial signature image is subjected to signature recognition to obtain the signature text recognition result; When the signature text recognition result matches the standard signature text, the initial signature image is determined as the standard signature image, and the initial signature text is determined as the standard signature text.

10. The method according to claim 8 or 9, characterized in that, Before verifying the electronic handwritten signature of the image to be verified and the standard signature image based on the signature verification model, the method further includes: In response to a trigger operation on the verification control, a signature interface is displayed, wherein the verification control is used to trigger the acquisition of the signature image to be verified; In response to a signature operation on the signature interface, the signature image to be verified is obtained; After determining the verification result of the signature image to be verified, the method further includes: When the verification result indicates that the target signature text matches the standard signature text and the target similarity is greater than the similarity threshold, target processing is performed. The target processing includes at least one of account login, information update, and account status unblocking.

11. A training device for a signature verification model, characterized in that, include: The sample acquisition module is used to acquire training samples, wherein the training samples include label images, sample images, signature labels of the sample images, and similar labels between the label images and the sample images, wherein the label images and the sample images are both electronic handwritten signature images; The model verification module is used to perform the following processing based on the model to be trained: extracting the first sample features of the sample image and the second sample features of the label image, performing signature recognition based on the first sample features to obtain signature recognition information, and determining similarity prediction information based on the first sample features and the second sample features; The model training module fuses the difference between the signature recognition information and the signature label with a first difference weight to obtain a first target difference; The difference between the similarity prediction information and the similar labels is fused with a second difference weight to obtain the second target difference; By combining the first target difference and the second target difference, the model to be trained is trained to obtain the signature verification model, wherein the signature verification model is used to verify the electronic handwritten signature of the signature image to be verified.

12. The apparatus according to claim 11, characterized in that, The model training module is also used for: During the training of the model to be trained, test samples are obtained; The electronic handwritten signature of the test sample is verified based on the currently trained model to obtain the signature accuracy and similarity accuracy. When the signature accuracy is greater than the similarity accuracy, at least one of the first difference weight and the second difference weight is adjusted to obtain a first target difference weight that is less than the second target difference weight. When the signature accuracy is less than the similarity accuracy, at least one of the first difference weight and the second difference weight is adjusted to obtain a first target difference weight that is greater than the second target difference weight. Based on the first target difference weight and the second target difference weight, the target loss for training the model to be trained is obtained.

13. A training device for a signature verification model, characterized in that, The training equipment includes: The first memory is used to store executable computer-executable instructions; The first processor, when executing executable computer-executable instructions stored in the first memory, implements the training method of the signature verification model according to any one of claims 1 to 7.

14. A data processing device based on signature verification, characterized in that, The data processing device includes: The second memory is used to store executable instructions for the computer; The second processor, when executing computer-executable instructions stored in the second memory, implements the signature-based data processing method according to any one of claims 8 to 10.

15. A computer-readable storage medium storing computer-executable instructions, characterized in that, When executed by a first processor, the computer-executable instructions implement the training method of the signature verification model according to any one of claims 1 to 7; or, when executed by a second processor, the executable instructions implement the data processing method based on signature verification according to any one of claims 8 to 10.

16. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by the first processor, they implement the training method of the signature verification model according to any one of claims 1 to 7; or, when the computer program or computer-executable instructions are executed by the second processor, they implement the data processing method based on signature verification according to any one of claims 8 to 10.

Images