Handwritten signature authentication method based on multi-task learning

Abstract

The invention relates to a handwritten signature authentication method based on multi-task learning, and the method comprises the steps: carrying out the multi-task learning through a handwritten signature verification task and an optical character recognition task, carrying out the learning of the text information of a handwriting picture through the optical character recognition task, and carrying out the judgment of the handwritten signature verification task and the auxiliary handwritten signature verification task. Comprising the following steps: extracting text information from an offline signature verification picture; a Laplacian attention module is constructed; a handwritten signature verification task and optical character recognition task sharing parameter module is constructed and is used for extracting common signature image features of the two tasks; constructing a first independent parameter module for an optical character recognition task, wherein the module is a Mamba module; constructing a second independent parameter module for a handwritten signature verification task, wherein the module is a multi-scale global fusion module; and a classification model is constructed, and after output features are integrated, whether the two pictures are similar or not is judged through the classification model. The method is beneficial to improving the accuracy of handwritten signature authentication.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and pattern recognition, and in particular to a handwritten signature authentication method based on multi-task learning. Background Art

[0002] In today's society, signature handwriting, as one of the important forensic means, is widely used in many fields such as law, insurance, and culture. Due to the uniqueness, stability, and reliability of signature handwriting, it has become an important basis for identifying the authenticity of documents and confirming the identity of the parties. However, with the continuous development of science and technology, signature handwriting inspection also faces many challenges. Signature handwriting is easily affected by writing habits, emotions, environment and other factors, making the accuracy of handwriting inspection complicated. In addition, the development of signature handwriting imitation technology has also brought certain difficulties to the inspection work. In view of the above challenges, it is necessary to further explore new signature handwriting inspection methods to improve the accuracy of handwritten signature verification. Summary of the invention

[0003] The purpose of the present invention is to provide a handwritten signature authentication method based on multi-task learning, which is conducive to improving the accuracy of handwritten signature authentication.

[0004] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a handwritten signature authentication method based on multi-task learning, which performs multi-task learning through a handwritten signature verification task and an optical character recognition task, and learns the text information of a handwriting image through the optical character recognition task, which is used for the handwritten signature verification task and assists in the judgment of the handwritten signature verification task, and includes the following steps: Step S1, extracting text information from an offline signature verification picture in a handwritten signature verification data set, and establishing a handwritten signature verification picture and text data set including an offline signature verification picture and corresponding text information; Step S2: construct a Laplace attention module, use the edge information extracted by the Laplace operator to weight the original image, and highlight the edge information of the image; Step S3: Based on the constructed Laplace attention module, a shared parameter module for the handwritten signature verification task and the optical character recognition task is constructed. The shared parameter module is used to extract the signature image features common to the handwritten signature verification task and the optical character recognition task. , where H, W and C represent the height, width and number of channels of the feature map respectively; Step S4, constructing a first independent parameter module for optical character recognition task, the first independent parameter module is a Mamba module, using the Mamba structure to reprocess the signature image features extracted in step S3; using the Mamba module to extract the relationship between the front and back and state spaces between the image sequences, and then using the connection time series classification algorithm to classify the characters; Step S5: construct a second independent parameter module for the handwritten signature verification task. The second independent parameter module is a multi-scale global fusion module, which is composed of a global attention module and a convolutional layer. The second independent parameter module is used to extract image features at different scales to determine the similarity between images. The signature image features extracted in step S3 are output after passing through the multi-scale global fusion module. ; Step S6: Build a classification model and output feature F from step S5 MGFB After the height, width, and number of channels are integrated into H×W×C, the classification model outputs a unique probability value to determine whether two images are similar.

[0005] Furthermore, in step S1, for the offline signature verification picture in the handwritten signature verification data set, the text information in the picture is extracted through OCR technology or open source text of the data set, and the text information corresponds to the offline signature verification picture one-to-one, thereby establishing a handwritten signature verification picture and text data set containing the offline signature verification picture and the corresponding text information.

[0006] Furthermore, in step S2, the implementation method of the Laplace attention module is: The Laplace operator is designed as: For the input image, after passing through the Laplacian operator, an edge feature map is obtained, the edge feature map is upsampled using the nearest neighbor algorithm, and a convolution operation is performed using sigmoid activation to obtain the convolution feature g; assuming the original image is h, in the Laplacian attention module, h is multiplied by g and then added to h to obtain the intermediate process attention h·g+h; then, the intermediate attention measurement is received through the global average pooling layer and the fully connected layer with sigmoid activation and the weight vector f is output; each channel is multiplied by the intermediate attention measurement of each element of f to generate the final attention mask (h·g+h)×f, which is the output of the Laplacian attention module.

[0007] Furthermore, in step S3, the shared parameter module of the handwritten signature verification task and the optical character recognition task consists of two convolution modules, each convolution module consists of six parts: a convolution layer, a ReLU layer, a Laplace attention module, a convolution layer, a ReLU layer, and a maximum pooling layer. The convolution kernel size of the convolution layer is 3, and the step size is 1; the pooling kernel size of the pooling layer is 2, and the step size is 2; the number of channels of the first convolution module is 64 channels, and the number of channels of the second convolution module is 128 channels; the shared parameter module is used to extract the common image edge information features of the handwritten signature verification task and the optical character recognition task.

[0008] Furthermore, in step S4, in order to obtain the correspondence between the front and back of the text information, the two-dimensional image features are serialized, and the Mamba structure is used to extract the front and back and state space of the sequence, and finally the connection time series classification algorithm is used to classify and splice the characters; specifically: First, the features of the two-dimensional image are serialized. According to the height and width of the image features, it can be divided into H·W grids. From left to right and from top to bottom, the image features are converted from three-dimensional features of H×W×C to two-dimensional features of [H×W]×C. Then, the Mamba module is used to extract the features of the front and back relationships. The formula of the state space model in the Mamba module is described as follows: Among them, h(t) is the current state quantity, A is the state transfer matrix; x(t) is the input control quantity, B represents the influence of the control quantity on the state quantity; y(t) represents the output of the system, C represents the influence of the current state quantity on the output, and D represents the influence of the current control quantity on the output; Then, the connection time sequence classification algorithm is used to classify and splice the characters. The formula of the connection time sequence classification algorithm is described as follows: Given an input sequence X=(x1, x2, ..., x T ), the CTC goal is to maximize the output sequence L=(l1, l2,..., l U ) probability P(L|X); this probability is obtained by summing the probabilities of all intermediate sequences Y that can be decoded into L: Among them, δ(Y) represents the process of mapping the intermediate sequence Y to the label sequence L, that is, removing blank symbols and merging consecutive identical labels; For each path Y, its probability P(Y|X) is calculated based on the output probability p(y i |x i ) is calculated; CTC assumes that the outputs of each time step are independent, so: .

[0009] Further, in step S5, the multi-scale global fusion module is composed of two 3×3 convolutional layers and one 2×2 maximum pooling layer; when two test images (img1, img2) are output, the test images (img1, img2) are the signature image features extracted in step S3, and the two images are first channel-fused into the first fusion image Fusion1, and then divided into three-layer paths for processing; the first layer path first passes through the global attention module to process the global information, and then extracts features through the first convolutional layer, and outputs the feature Output CNN1 ; Then, the feature Output of the first convolutional layer CNN1 It is fused with the first fusion image Fusion1 to form the second fusion image Fusion2, which is then passed through the global attention module and the second convolutional layer to output the second layer feature Output. CNN2 ; Finally, the first fusion image Fusion1, the second fusion image Fusion2 and the second layer feature Output CNN2 After fusion and maximum pooling, the final features are output; the specific formula is as follows: Among them, img1 and img2 are two pictures to be tested. represents channel fusion, GA represents global attention, CNN represents 3x3 convolutional layer, and MaxPool represents the maximum pooling layer.

[0010] Furthermore, the formula of global attention is expressed as follows: Among them, Q, K, V are the linear projections of self-attention, and d is the bias constant.

[0011] Furthermore, in step S6, the classification model consists of three linear layers, two ReLU layers and a sigmoid function, wherein the output features of the multi-scale global fusion module are converted from H×W×C to [1×C] through the global pooling function, and then enter the C×512, 512×258, and 258×1 linear layers respectively, wherein the linear layers contain ReLU layers, and finally the final probability value is obtained through the sigmoid function.

[0012] Further, on the basis of constructing the Laplace attention module in step S2, a parameter sharing module for the handwritten signature verification task and the optical character recognition task is constructed in step S3; the optical character recognition task is implemented twice based on the modules constructed in steps S2, S3, and S4, and the handwritten signature verification task is implemented based on the modules constructed in steps S2, S3, and S5, and then an end-to-end handwritten signature authentication model based on multi-task learning is established in combination with the classification model constructed in step S6; The end-to-end handwritten signature authentication model is trained using the handwritten signature verification image and text dataset established in step S1; finally, the handwritten signature authentication is performed using the trained end-to-end handwritten signature authentication model.

[0013] Furthermore, the loss function of the end-to-end handwritten signature verification model consists of the following parts: the loss function of the two optical character recognition tasks Loss OCR1 、Loss OCR2 , a loss function Loss for a handwritten signature verification task HSV ; Therefore, the overall loss function Loss is: Among them, α is a hyperparameter.

[0014] Compared with the prior art, the present invention has the following beneficial effects: the present invention provides a handwritten signature authentication method based on multi-task learning, which performs multi-task learning through a handwritten signature verification task and an optical character recognition task, learns text information of a handwriting image through the optical character recognition task, and uses it for the handwritten signature verification task to assist in the judgment of the handwritten signature verification task, reduce the overfitting of the handwritten signature verification task, and improve the accuracy of the handwritten signature verification task. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention; Figure 2 It is a block diagram of the implementation principle of an end-to-end handwritten signature authentication model based on multi-task learning in an embodiment of the present invention; Figure 3 is a block diagram of the implementation principle of the Laplace attention module in an embodiment of the present invention; Figure 4 is an extracted feature map of the Laplace attention module in an embodiment of the present invention; Figure 5 It is a block diagram of the implementation principle of the multi-scale global fusion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0017] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0018] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0019] This embodiment provides a handwritten signature authentication method based on multi-task learning, which performs multi-task learning through handwritten signature verification tasks and optical character recognition tasks, and learns the text information of handwriting images through the optical character recognition task, which is used for the handwritten signature verification task to assist in the judgment of the handwritten signature verification task. Figure 1 As shown, the method specifically comprises the following steps: Step S1: extract text information from an offline signature verification picture in a handwritten signature verification data set, and establish a handwritten signature verification picture and text data set including the offline signature verification picture and corresponding text information.

[0020] Step S2: construct a Laplace attention module, use the edge information extracted by the Laplace operator to weight the original image, and highlight the edge information of the image.

[0021] Step S3: Based on the constructed Laplace attention module, a shared parameter module for the handwritten signature verification task and the optical character recognition task is constructed. The shared parameter module is used to extract the signature image features common to the handwritten signature verification task and the optical character recognition task. The signature image features are expressed as , where H, W, and C represent the height, width, and number of channels of the feature map, respectively.

[0022] Step S4, constructing a first independent parameter module for optical character recognition task, the first independent parameter module is a Mamba module, using the Mamba structure to reprocess the signature image features extracted in step S3; using the Mamba module to extract the relationship between the front and back and state spaces between the image sequences, and then using the connection time series classification algorithm to classify the characters. Before the Mamba module, the height and width of the signature image features are flattened, that is, I=H·W.

[0023] Step S5: construct a second independent parameter module for the handwritten signature verification task. The second independent parameter module is a multi-scale global fusion module, which is composed of a global attention module and a convolutional layer. The second independent parameter module is used to extract image features at different scales to determine the similarity between images. The signature image features extracted in step S3 are output after passing through the multi-scale global fusion module. , where H, W and C represent the height, width and number of channels of the output feature map of the multi-scale global fusion module respectively.

[0024] Step S6: Build a classification model and output feature F from step S5 MGFB After the height, width, and number of channels are integrated into H×W×C, the classification model outputs a unique probability value to determine whether two images are similar.

[0025] On the basis of constructing the Laplace attention module in step S2, a parameter sharing module for the handwritten signature verification task and the optical character recognition task is constructed in step S3; the optical character recognition task is implemented twice based on the modules constructed in steps S2, S3, and S4, and the handwritten signature verification task is implemented based on the modules constructed in steps S2, S3, and S5, and then an end-to-end handwritten signature authentication model based on multi-task learning is established in combination with the classification model constructed in step S6. In this embodiment, the end-to-end handwritten signature authentication model based on multi-task learning is as follows: Figure 2 shown.

[0026] The end-to-end handwritten signature authentication model is trained using the handwritten signature verification image and text dataset established in step S1; finally, the handwritten signature authentication is performed using the trained end-to-end handwritten signature authentication model.

[0027] In step S1, for the offline signature verification picture in the handwritten signature verification data set, the text information in the picture is extracted by means of OCR technology or open source text of the data set, and the text information corresponds to the offline signature verification picture one by one, thereby establishing a handwritten signature verification picture and text data set containing the offline signature verification picture and the corresponding text information.

[0028] The Laplace operator has the function of extracting edge information. In step S2, the implementation method of the Laplace attention module is: The Laplace operator is designed as: For the input image, after passing through the Laplacian operator, an edge feature map is obtained, the edge feature map is upsampled using the nearest neighbor algorithm, and a convolution operation is performed using sigmoid activation to obtain the convolution feature g; assuming the original image is h, in the Laplacian attention module, h is multiplied by g and then added to h to obtain the intermediate process attention h·g+h, where "·" represents element-wise multiplication; then, the intermediate attention measurement is received through the global average pooling layer (GAP) and the fully connected layer (FC) with sigmoid activation and the weight vector f is output; each channel is multiplied by the intermediate attention measurement of each element of f to generate the final attention mask (h·g+h)×f, which is the output of the Laplacian attention module.

[0029] In this embodiment, the structure and implementation process of the Laplace attention module are as follows: Figure 3 As shown in Figure 4 shown.

[0030] In step S3, the parameter module shared by the handwritten signature verification task and the optical character recognition task consists of two convolution modules, each of which consists of six parts: a convolution layer, a ReLU layer, a Laplace attention module, a convolution layer, a ReLU layer, and a maximum pooling layer. The convolution kernel size of the convolution layer is 3, the step size is 1, the pooling kernel size of the pooling layer is 2, and the step size is 2; the number of channels of the first convolution module is 64 channels, and the number of channels of the second convolution module is 128 channels; the shared parameter module is used to extract the common image edge information features of the handwritten signature verification task and the optical character recognition task.

[0031] Mamba is a state-space model with linear computational complexity, which aims to solve the computational complexity and memory usage problems faced by traditional Transformers when processing long sequences. In step S4, in order to obtain the correspondence between the front and back of text information, the two-dimensional image features need to be serialized, and the Mamba structure is used to extract the front and back of the sequence and the state space, and finally the connection temporal classification algorithm is used to classify and splice the characters; specifically: First, the features of the two-dimensional image are serialized. According to the height and width of the image features, it can be divided into H·W grids. From left to right and from top to bottom, the image features are converted from three-dimensional features of H×W×C to two-dimensional features of [H×W]×C. Then, the Mamba module is used to extract the features of the front and back relationships. The formula of the state space model in the Mamba module is described as follows: Among them, h(t) is the current state quantity, A is the state transfer matrix; x(t) is the input control quantity, B represents the influence of the control quantity on the state quantity; y(t) represents the output of the system, C represents the influence of the current state quantity on the output, and D represents the influence of the current control quantity on the output.

[0032] Then, the connection time sequence classification algorithm is used to classify and splice the characters. The formula of the connection time sequence classification algorithm is described as follows: Given an input sequence X=(x1, x2, ..., x T ), the CTC goal is to maximize the output sequence L=(l1, l2,..., l U ) probability P(L|X); this probability is obtained by summing the probabilities of all intermediate sequences Y that can be decoded into L: Among them, δ(Y) represents the process of mapping the intermediate sequence Y to the label sequence L, that is, removing blank symbols and merging consecutive identical labels.

[0033] For each path Y, its probability P(Y|X) is calculated based on the output probability p(y i |x i ) is calculated; CTC assumes that the outputs of each time step are independent, so: .

[0034] In this embodiment, the multi-scale global fusion module is as follows: Figure 5 As shown. Figure 5 As shown, in step S5, the multi-scale global fusion module consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer; when two test images (img1, img2) are output, the test images (img1, img2) are the signature image features extracted in step S3, and the two images are first channel-fused into the first fusion image Fusion1, and then divided into three-layer paths for processing; the first layer path first passes through the global attention module to process the global information, and then extracts features through the first convolutional layer, and outputs the feature Output CNN1 ; Then, the feature Output of the first convolutional layer CNN1 It is fused with the first fusion image Fusion1 to form the second fusion image Fusion2, which is then passed through the global attention module and the second convolutional layer to output the second layer feature Output. CNN2 ; Finally, the first fusion image Fusion1, the second fusion image Fusion2 and the second layer feature Output CNN2 After fusion and maximum pooling, the final features are output; the specific formula is as follows: Among them, img1 and img2 are two pictures to be tested. represents channel fusion, GA represents global attention, CNN represents 3x3 convolutional layer, and MaxPool represents the maximum pooling layer.

[0035] Among them, the formula of global attention is expressed as follows: Among them, Q, K, V are the linear projections of self-attention, and d is the bias constant.

[0036] In step S6, the classification model consists of three linear layers, two ReLU layers and a sigmoid function, where the output features of the multi-scale global fusion module are converted from H×W×C to [1×C] through the global pooling function, and then enter the C×512, 512×258, and 258×1 linear layers respectively, where the linear layers contain ReLU layers, and finally the final probability value is obtained through the sigmoid function.

[0037] In this embodiment, the loss function of the end-to-end handwritten signature verification model consists of the following parts: the loss function of the two optical character recognition tasks Loss OCR1 、Loss OCR2 , a loss function Loss for a handwritten signature verification task HSV .

[0038] Therefore, the overall loss function Loss is: Among them, α is a hyperparameter.

[0039] This embodiment also provides a handwritten signature authentication system based on multi-task learning, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method can be implemented.

[0040] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0041] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0042] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0044] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A handwritten signature authentication method based on multi-task learning, characterized in that: Multi-task learning is performed through handwritten signature verification tasks and optical character recognition tasks. The text information of the handwriting image is learned through the optical character recognition task for the handwritten signature verification task to assist in the judgment of the handwritten signature verification task, including the following steps: Step S1, extracting text information from an offline signature verification picture in a handwritten signature verification data set, and establishing a handwritten signature verification picture and text data set including an offline signature verification picture and corresponding text information; Step S2: construct a Laplace attention module, use the edge information extracted by the Laplace operator to weight the original image, and highlight the edge information of the image; Step S3: Based on the constructed Laplace attention module, a shared parameter module for the handwritten signature verification task and the optical character recognition task is constructed. The shared parameter module is used to extract the signature image features common to the handwritten signature verification task and the optical character recognition task. , where H, W and C represent the height, width and number of channels of the feature map respectively; Step S4, constructing a first independent parameter module for optical character recognition task, the first independent parameter module is a Mamba module, using the Mamba structure to reprocess the signature image features extracted in step S3; using the Mamba module to extract the relationship between the front and back and state spaces between the image sequences, and then using the connection time series classification algorithm to classify the characters; Step S5: construct a second independent parameter module for the handwritten signature verification task. The second independent parameter module is a multi-scale global fusion module, which is composed of a global attention module and a convolutional layer. The second independent parameter module is used to extract image features at different scales to determine the similarity between images. The signature image features extracted in step S3 are output after passing through the multi-scale global fusion module. ; Step S6: Build a classification model and output feature F from step S5 MGFB After the height, width, and number of channels are integrated into H×W×C, the classification model outputs a unique probability value to determine whether two images are similar.

2. According to the handwritten signature authentication method based on multi-task learning in claim 1, it is characterized in that: In step S1, for the offline signature verification picture in the handwritten signature verification data set, the text information in the picture is extracted through OCR technology or open source text of the data set, and the text information corresponds to the offline signature verification picture one by one, thereby establishing a handwritten signature verification picture and text data set containing the offline signature verification picture and the corresponding text information.

3. According to the handwritten signature authentication method based on multi-task learning in claim 1, it is characterized in that: In step S2, the implementation method of the Laplace attention module is: The Laplace operator is designed as: For the input image, after passing through the Laplacian operator, an edge feature map is obtained, the edge feature map is upsampled using the nearest neighbor algorithm, and a convolution operation is performed using sigmoid activation to obtain the convolution feature g; assuming the original image is h, in the Laplacian attention module, h is multiplied by g and then added to h to obtain the intermediate process attention h·g+h; then, the intermediate attention measurement is received through the global average pooling layer and the fully connected layer with sigmoid activation and the weight vector f is output; each channel is multiplied by the intermediate attention measurement of each element of f to generate the final attention mask (h·g+h)×f, which is the output of the Laplacian attention module.

4. According to the handwritten signature authentication method based on multi-task learning in claim 1, it is characterized in that: In step S3, the parameter module shared by the handwritten signature verification task and the optical character recognition task consists of two convolution modules, each of which consists of six parts: a convolution layer, a ReLU layer, a Laplace attention module, a convolution layer, a ReLU layer, and a maximum pooling layer. The convolution kernel size of the convolution layer is 3, the step size is 1, the pooling kernel size of the pooling layer is 2, and the step size is 2; the number of channels of the first convolution module is 64 channels, and the number of channels of the second convolution module is 128 channels; the shared parameter module is used to extract the common image edge information features of the handwritten signature verification task and the optical character recognition task.

5. According to the handwritten signature authentication method based on multi-task learning in claim 1, it is characterized in that: In step S4, in order to obtain the correspondence between the front and back of the text information, the two-dimensional image features are serialized, and the Mamba structure is used to extract the front and back and state space of the sequence, and finally the connection time series classification algorithm is used to classify and splice the characters; specifically: First, the features of the two-dimensional image are serialized. According to the height and width of the image features, it can be divided into H·W grids. From left to right and from top to bottom, the image features are converted from three-dimensional features of H×W×C to two-dimensional features of [H×W]×C. Then, the Mamba module is used to extract the features of the front and back relationships. The formula of the state space model in the Mamba module is described as follows: Among them, h(t) is the current state quantity, A is the state transfer matrix; x(t) is the input control quantity, B represents the influence of the control quantity on the state quantity; y(t) represents the output of the system, C represents the influence of the current state quantity on the output, and D represents the influence of the current control quantity on the output; Then, the connection time sequence classification algorithm is used to classify and splice the characters. The formula of the connection time sequence classification algorithm is described as follows: Given an input sequence X=(x1, x2, ..., x T ), the CTC goal is to maximize the output sequence L=(l1, l2,..., l U ) probability P(L|X); this probability is obtained by summing the probabilities of all intermediate sequences Y that can be decoded into L: Among them, δ(Y) represents the process of mapping the intermediate sequence Y to the label sequence L, that is, removing blank symbols and merging consecutive identical labels; For each path Y, its probability P(Y|X) is calculated based on the output probability p(y i |x i ) is calculated; CTC assumes that the outputs of each time step are independent, so: 。 6. According to the handwritten signature authentication method based on multi-task learning in claim 1, it is characterized in that: In step S5, the multi-scale global fusion module consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer; when two test images (img1, img2) are output, the test images (img1, img2) are the signature image features extracted in step S3, and the two images are first channel-fused into the first fusion image Fusion1, and then divided into three-layer paths for processing; the first layer path first passes through the global attention module to process global information, and then extracts features through the first convolutional layer, and outputs the feature Output CNN1 ; Then, the feature Output of the first convolutional layer CNN1 It is fused with the first fusion image Fusion1 to form the second fusion image Fusion2, which is then passed through the global attention module and the second convolutional layer to output the second layer feature Output. CNN2 ; Finally, the first fusion image Fusion1, the second fusion image Fusion2 and the second layer feature Output CNN2 Fusion, after maximum pooling, output the final features; The specific formula is as follows: Among them, img1 and img2 are two pictures to be tested. represents channel fusion, GA represents global attention, CNN represents 3x3 convolutional layer, and MaxPool represents the maximum pooling layer.

7. A handwritten signature authentication method based on multi-task learning according to claim 6, characterized in that: The formula of global attention is as follows: Among them, Q, K, V are the linear projections of self-attention, and d is the bias constant.

8. The handwritten signature authentication method based on multi-task learning according to claim 1 is characterized in that: In step S6, the classification model consists of three linear layers, two ReLU layers and a sigmoid function, where the output features of the multi-scale global fusion module are converted from H×W×C to [1×C] through the global pooling function, and then enter the C×512, 512×258, and 258×1 linear layers respectively, where the linear layers contain ReLU layers, and finally the final probability value is obtained through the sigmoid function.

9. The handwritten signature authentication method based on multi-task learning according to claim 1 is characterized in that: On the basis of constructing the Laplace attention module in step S2, constructing a parameter sharing module for the handwritten signature verification task and the optical character recognition task in step S3; implementing the optical character recognition task twice based on the modules constructed in steps S2, S3, and S4, implementing the handwritten signature verification task based on the modules constructed in steps S2, S3, and S5, and then establishing an end-to-end handwritten signature authentication model based on multi-task learning in combination with the classification model constructed in step S6; The end-to-end handwritten signature authentication model is trained using the handwritten signature verification image and text dataset established in step S1; Finally, the trained end-to-end handwritten signature authentication model is used to perform handwritten signature authentication.

10. A handwritten signature authentication method based on multi-task learning according to claim 9, characterized in that: The loss function of the end-to-end handwritten signature verification model consists of the following parts: Loss function of two optical character recognition tasks OCR1 、Loss OCR2 , a loss function Loss for a handwritten signature verification task HSV ; Therefore, the overall loss function Loss is: Among them, α is a hyperparameter.

Images