A Chinese Handwritten Signature Recognition Method and System Based on Deep Learning

Abstract

The present invention discloses a Chinese handwritten signature recognition method and system based on deep learning. The method includes: generating a training data set including Chinese handwritten signature pictures; processing the Chinese handwritten signature pictures to generate an enhanced data set; training a Chinese signature detection algorithm model through the enhanced data set. When the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification results of Chinese handwritten signatures respectively reach the thresholds, the Chinese signature detection algorithm model is used as the final Chinese signature detection algorithm model; collecting Chinese handwritten signature pictures to be recognized and preprocessing the Chinese handwritten signature pictures to be recognized; recognizing the preprocessed Chinese handwritten signature pictures to be recognized through the final Chinese signature detection algorithm model and outputting the border coordinates and classification results of the Chinese handwritten signatures in the Chinese handwritten signature pictures to be recognized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically, to a Chinese handwritten signature recognition method and system based on deep learning. Background Art

[0002] Handwritten signatures can be seen everywhere in daily life. Simply put, it is writing one's own name by hand. Using a handwritten signature on a paper document is mainly used to determine the identity of the signer, and indicates that the signer agrees to the content stipulated in the signed document, is responsible for the authenticity of the document, and has legal effect. Thus, the importance of handwritten signatures can be seen. The existing technology has low accuracy and low recognition efficiency for Chinese handwritten signature recognition in electronic documents.

[0003] Existing Technology 1 (Publication No. CN110008909A) classifies the input signature pictures by establishing a binary classification model based on deep learning, into valid signatures or invalid signatures. However, Existing Technology 1 classifies the signature pictures based on the known signature position to determine whether it is a valid signature. Existing Technology 1 cannot solve the problem of recognizing signatures with unknown signature positions.

[0004] Existing Technology 2 (Publication No. CN113095203A) uses existing video parsing technology to parse the video to obtain multiple consecutive pictures as a signature picture group, then uses the YoloV4 algorithm to detect the signature picture group to obtain the signature pen recognition result, and then uses the Faster R-CNN algorithm to detect to obtain the recognition result, and combines the two to judge whether the signature passes. Existing Technology 2 still needs to separately judge whether the signature is valid, and the recognition efficiency is low.

[0005] Therefore, a technology is needed to realize the recognition of Chinese handwritten signatures based on a deep learning model. Summary of the Invention

[0006] The technical solution of the present invention provides a Chinese handwritten signature recognition method and system based on deep learning to solve the problem of how to recognize Chinese handwritten signatures based on deep learning.

[0007] To solve the above problems, the present invention provides a Chinese handwritten signature recognition method based on deep learning, and the method includes:

[0008] Generating a training data set including Chinese handwritten signature pictures;

[0009] Processing the Chinese handwritten signature pictures to generate an enhanced data set;

[0010] Train the Chinese signature detection algorithm model with the enhanced dataset. When the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification results of Chinese handwritten signatures reach the thresholds respectively, use the Chinese signature detection algorithm model as the final Chinese signature detection algorithm model;

[0011] Collect the Chinese handwritten signature pictures to be recognized, and preprocess the Chinese handwritten signature pictures to be recognized;

[0012] Recognize the preprocessed Chinese handwritten signature pictures to be recognized through the final Chinese signature detection algorithm model, and output the border coordinates and classification results of the Chinese handwritten signatures in the Chinese handwritten signature pictures to be recognized.

[0013] Preferably, the generation of the training dataset including Chinese handwritten signature pictures includes:

[0014] Set the size of the picture and the size of the font;

[0015] Randomly select a background from the background library, randomly select a name from the name library, and randomly select a font from the font library;

[0016] Use the background as the background of the picture, add the name to the picture according to the font and the size of the font, and generate Chinese handwritten signature pictures.

[0017] Preferably, the processing of the Chinese handwritten signature pictures to generate an enhanced dataset includes at least one of the following processing methods:

[0018] Perform partial random erasure on the Chinese handwritten signature pictures;

[0019] Scale the Chinese handwritten signature pictures to a first preset size, and then paste them onto blank pictures of a second preset size; the second preset size is larger than the first preset size;

[0020] Perform random cropping or random flipping on the Chinese handwritten signature pictures;

[0021] Perform random angular rotation on the Chinese handwritten signature pictures, where θ represents the rotation angle, and the value range is θ ∈ (-5, 5);

[0022] Perform random brightness, contrast, and hue adjustment on the Chinese handwritten signature pictures;

[0023] Perform random noise adjustment on the Chinese handwritten signature pictures;

[0024] Perform random Gaussian blur adjustment on the Chinese handwritten signature pictures;

[0025] Randomly mix multiple of the Chinese handwritten signature images through an open-source Mosaic enhancement method to obtain a new Chinese handwritten signature image.

[0026] Preferably, the training of the Chinese signature detection algorithm model using the enhancement dataset further includes:

[0027] Using the open-source EfficientNet as the base network and combining with the open-source PANet to extract features from the Chinese handwritten signature images;

[0028] Extract the bounding box coordinates of the Chinese signature contained in the Chinese handwritten signature image through the detection algorithm;

[0029] Calculate the probability that the detected bounding box coordinates are the bounding box of the Chinese signature through the classification algorithm;

[0030] Calculate the bounding box gap between the true Chinese signature bounding box and the detected bounding box coordinates based on the smooth L1 loss function to obtain the bounding box gap accuracy;

[0031] Calculate the classification gap between the predicted classification result and the true result based on the cross-entropy loss function to obtain the classification gap accuracy;

[0032] When the bounding box gap accuracy and the classification gap accuracy respectively reach the thresholds, stop training the Chinese signature detection algorithm model.

[0033] Preferably, the collection of the Chinese handwritten signature image to be recognized and the preprocessing of the Chinese handwritten signature image to be recognized include:

[0034] Collect the Chinese handwritten signature image to be recognized through a mobile terminal, a PC, or an image acquisition device, or upload the Chinese handwritten signature image to be recognized from an album or a gallery; the color channels of the Chinese handwritten signature image to be recognized include color or grayscale, and the direction is positive upward;

[0035] Scale the width and height dimensions of the Chinese handwritten signature image to be recognized proportionally to 640×640, and normalize all pixel values to between 0 and 1 after dividing by 255.

[0036] Preferably, the output of the bounding box coordinates and the classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized includes:

[0037] Output the bounding box coordinates of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the center point coordinates of the bounding box, Width is the width of the bounding box, and Height is the height of the bounding box;

[0038] Output the classification result of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized. The classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, retain the border coordinates of the Chinese handwritten signature picture to be recognized.

[0039] Based on another aspect of the present invention, the present invention provides a Chinese handwritten signature recognition system based on deep learning. The system includes:

[0040] A training unit for generating a training data set including Chinese handwritten signature pictures; processing the Chinese handwritten signature pictures to generate an enhanced data set; training a Chinese signature detection algorithm model through the enhanced data set. When the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification results of Chinese handwritten signatures respectively reach the threshold, use the Chinese signature detection algorithm model as the final Chinese signature detection algorithm model;

[0041] An identification unit for collecting a Chinese handwritten signature picture to be recognized and preprocessing the Chinese handwritten signature picture to be recognized; recognizing the preprocessed Chinese handwritten signature picture to be recognized through the final Chinese signature detection algorithm model, and outputting the border coordinates and classification results of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized.

[0042] Preferably, the training unit is used to generate a training data set including Chinese handwritten signature pictures, including:

[0043] Set the size of the picture and the size of the font;

[0044] Randomly select a background based on a background library, randomly select a person's name based on a person's name library, and randomly select a font based on a font library;

[0045] Use the background as the background of the picture, add the person's name to the picture according to the font and the size of the font, and generate a Chinese handwritten signature picture.

[0046] Preferably, the training unit is used to process the Chinese handwritten signature pictures to generate an enhanced data set, including at least one of the following processing methods:

[0047] Perform partial random erasure on the Chinese handwritten signature picture;

[0048] Scale the Chinese handwritten signature picture to a first preset size and then paste it onto a blank picture of a second preset size; the second preset size is larger than the first preset size;

[0049] Perform random cropping or random flipping on the Chinese handwritten signature picture;

[0050] Randomly rotate the Chinese handwritten signature image by an angle θ, where the range of θ is θ ∈ (-5, 5);

[0051] Adjust the random brightness, contrast, and hue of the Chinese handwritten signature image;

[0052] Adjust the random noise of the Chinese handwritten signature image;

[0053] Adjust the random Gaussian blur of the Chinese handwritten signature image;

[0054] Randomly mix multiple Chinese handwritten signature images as new Chinese handwritten signature images through the open-source Mosaic enhancement method.

[0055] Preferably, the training unit is used to train the Chinese signature detection algorithm model through the enhanced dataset, and further includes:

[0056] Based on the open-source EfficientNet as the basic network, and combined with the open-source PANet to extract features from the Chinese handwritten signature image;

[0057] Extract the bounding box coordinates of the Chinese signature contained in the Chinese handwritten signature image through the detection algorithm;

[0058] Calculate the probability that the detected bounding box coordinates are the bounding box of the Chinese signature through the classification algorithm;

[0059] Calculate the bounding box gap between the true Chinese signature bounding box and the detected bounding box coordinates based on the smooth L1 loss function, and obtain the bounding box gap accuracy;

[0060] Calculate the classification gap between the predicted classification result and the true result based on the cross-entropy loss function, and obtain the classification gap accuracy;

[0061] When the bounding box gap accuracy and the classification gap accuracy respectively reach the threshold, stop training the Chinese signature detection algorithm model.

[0062] Preferably, the recognition unit is used to collect the Chinese handwritten signature image to be recognized and preprocess the Chinese handwritten signature image to be recognized, including:

[0063] Collect the Chinese handwritten signature image to be recognized through a mobile terminal, a PC, or an image acquisition device, or upload the Chinese handwritten signature image to be recognized from an album or a gallery; the color channel of the Chinese handwritten signature image to be recognized includes color or grayscale, and the direction is positive upward;

[0064] The width and height dimensions of the Chinese handwritten signature image to be recognized are scaled proportionally to 640×640, and all pixel values are normalized to between 0 and 1 after being divided by 255.

[0065] Preferably, the recognition unit is used to output the border coordinates and classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, including:

[0066] Output the border coordinates of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the coordinates of the center point of the border, Width is the width of the border, and Height is the height of the border;

[0067] Output the classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized. The classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, the border coordinates of the Chinese handwritten signature image to be recognized are retained.

[0068] The technical solution of the present invention provides a Chinese handwritten signature recognition method and system based on deep learning. The method includes: generating a training data set including Chinese handwritten signature images; processing the Chinese handwritten signature images to generate an enhanced data set; training a Chinese signature detection algorithm model through the enhanced data set. When the recognition accuracy of the Chinese signature detection algorithm model for the border coordinates and classification result of the Chinese handwritten signature reaches the threshold respectively, the Chinese signature detection algorithm model is used as the final Chinese signature detection algorithm model; collecting the Chinese handwritten signature image to be recognized, and preprocessing the Chinese handwritten signature image to be recognized; recognizing the preprocessed Chinese handwritten signature image to be recognized through the final Chinese signature detection algorithm model, and outputting the border coordinates and classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized. The technical solution of the present invention combines the idea of artificial intelligence and uses computer vision technology to train handwritten signatures, obtaining a training accuracy of up to 100%. And after optimizing the training model, a set of handwritten signature recognition system is implemented. The technical solution of the present invention realizes detecting the position of the signature in an image with an unknown signature position. The technical solution of the present invention uses deep learning technology to design a dedicated Chinese signature detection algorithm, which can directly detect Chinese handwritten signatures in one step without the need for extra steps to judge whether the signature is valid, and can achieve a fast and efficient detection effect. Description of the Drawings

[0069] By referring to the following drawings, the exemplary embodiments of the present invention can be more completely understood:

[0070] Figure 1Flowchart of a deep - learning - based Chinese handwritten signature recognition method according to a preferred embodiment of the present invention;

[0071] Figure 2 Overall flowchart of a deep - learning - based Chinese handwritten signature detection method according to a preferred embodiment of the present invention;

[0072] Figure 3 Overall flowchart of the training stage according to a preferred embodiment of the present invention;

[0073] Figure 4 Flowchart of the dataset generation module in the training stage according to a preferred embodiment of the present invention;

[0074] Figure 5 Schematic diagram of a sample signature picture generated in the training stage according to a preferred embodiment of the present invention;

[0075] Figure 6 Schematic diagram of the overall algorithm model for Chinese signature detection in the training stage according to a preferred embodiment of the present invention;

[0076] Figure 7 Structural diagram of the detection / classification part of the Chinese signature detection algorithm in the training stage according to a preferred embodiment of the present invention;

[0077] Figure 8 Overall flowchart of the prediction stage according to a preferred embodiment of the present invention;

[0078] Figure 9 Picture obtained by the image acquisition module in the prediction stage according to a preferred embodiment of the present invention;

[0079] Figure 10 Flowchart of the final result output module in the prediction stage according to a preferred embodiment of the present invention;

[0080] Figure 11 Schematic diagram of the final detected signature box result according to a preferred embodiment of the present invention; and

[0081] Figure 12 Structural diagram of a deep - learning - based Chinese handwritten signature recognition system according to a preferred embodiment of the present invention. Specific embodiments

[0082] Reference is now made to the accompanying drawings to describe exemplary embodiments of the present invention. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to disclose the present invention in detail and completely, and to fully convey the scope of the present invention to those skilled in the art. The terms in the exemplary embodiments shown in the drawings are not intended to limit the present invention. In the drawings, the same units / components are denoted by the same reference numerals.

[0083] Unless otherwise specified, the terms used herein (including scientific and technical terms) have the ordinary meaning understood by those skilled in the art. Additionally, it can be understood that terms defined in commonly used dictionaries should be construed as having a meaning consistent with their context in the relevant art, and should not be construed as idealized or overly formal meanings.

[0084] Figure 1 It is a flowchart of a Chinese handwritten signature recognition based on deep learning according to a preferred embodiment of the present invention. The present invention proposes a method for detecting Chinese handwritten signatures based on deep learning, which is used to detect handwritten Chinese signatures and can be applied to various scenarios. Currently, there is no detection algorithm specifically designed for handwritten signatures on the market. The existing algorithms are mainly based on general object detection algorithms. However, compared with general objects, handwritten signature detection has the characteristics of a relatively small target, less obvious signature features, and being easily confused with ordinary document fonts. Therefore, general object detection algorithms generally have problems such as poor detection effect for handwritten signatures, long time consumption, and high hardware resource requirements.

[0085] The present invention proposes a method for detecting Chinese handwritten signatures based on deep learning, which fully considers the characteristics of handwritten signatures and specifically designs a detection algorithm, which can improve the accuracy of handwritten signatures, reduce time consumption, reduce hardware resource consumption, etc., and finally achieve efficient and accurate handwritten signature detection.

[0086] The present invention discloses a method for detecting Chinese handwritten signatures based on deep learning, which can improve the accuracy of handwritten signatures, reduce time consumption, reduce hardware resource consumption, etc., and achieve efficient and accurate handwritten signature detection. The present invention mainly consists of two parts: a training stage and a prediction stage. The overall flowchart is as Figure 1 and Figure 2 shown.

[0087] As Figure 1 shown, the present invention provides a method for recognizing Chinese handwritten signatures based on deep learning. The method includes:

[0088] Step 101: Generate a training data set including Chinese handwritten signature pictures;

[0089] Preferably, generating a training data set including Chinese handwritten signature pictures includes:

[0090] Set the size of the picture and the size of the font;

[0091] Randomly select a background from the background library, randomly select a name from the name library, and randomly select a font from the font library;

[0092] Use the background as the background of the picture, add the name to the picture according to the font and the size of the font, and generate a Chinese handwritten signature picture.

[0093] The training stage of the present invention mainly includes a dataset generation module, a dataset enhancement module, and a model training module. As shown in the appendix Figure 3 Shown is the overall flowchart of the training stage. The main purpose of the training stage is to continuously optimize the model and finally output a Chinese handwritten signature model with accurate detection.

[0094] The present invention generates a training dataset through a dataset generation module. With the rapid development of deep learning in the field of images and the various achievements it brings, the advantages of deep learning are becoming more and more prominent. The methods related to deep learning mainly rely on data. The more diverse the data is, the stronger the robustness and generalization ability of the learned model, and the higher the accuracy. Therefore, how to obtain more and more diverse data is also crucial. Chinese handwritten signatures face the problems of difficult access to image data and small quantity. The present invention designs a dataset generation method that can generate a large amount of and diverse datasets. The flowchart of the dataset generation module is as shown in the appendix Figure 4 Shown.

[0095] The present invention first collects a large number of background libraries, mainly including signature backgrounds in various complex scenarios; collects a large number of name libraries, mainly including two-character, three-character, and four-character names; collects a large number of font libraries, mainly including various handwritten signature fonts, imitating the effect of real signatures. Then randomly select a background from the background library, and then randomly select a name and a font from the name library and the font library to generate a signature. When the number of signatures on each background picture meets the requirements, the generation of this background picture is completed. The sample of the finally generated signature picture is as shown in the appendix Figure 5 Shown.

[0096] Step 102: Process the Chinese handwritten signature picture to generate an enhanced dataset;

[0097] Preferably, processing the Chinese handwritten signature picture to generate an enhanced dataset includes at least one of the following processing methods:

[0098] Perform partial random erasure on the Chinese handwritten signature picture;

[0099] Scale the Chinese handwritten signature image to the first preset size, and then paste it onto a blank image of the second preset size; the second preset size is larger than the first preset size;

[0100] Randomly crop or flip the Chinese handwritten signature image;

[0101] Randomly rotate the Chinese handwritten signature image by an angle, where θ represents the rotation angle and the value range is θ ∈ (-5, 5);

[0102] Randomly adjust the brightness, contrast, and hue of the Chinese handwritten signature image;

[0103] Randomly adjust the noise of the Chinese handwritten signature image;

[0104] Randomly perform Gaussian blur adjustment on the Chinese handwritten signature image;

[0105] Randomly mix multiple Chinese handwritten signature images as a new Chinese handwritten signature image through the open-source Mosaic enhancement method.

[0106] The present invention generates an enhanced dataset through a dataset enhancement module. In actual scenarios, signature images, such as in scenarios like taking pictures, may have problems such as wrinkles, occlusions, and blurs. And deep learning-based methods rely more on data diversity and authenticity, and the generated signature images are relatively clear and clean. Therefore, in order to make the generated signature images more realistic and better meet the real scenarios, the present invention designs a dataset enhancement module to make the signature images better meet the requirements of real scenarios. Under the action of the data enhancement module, the accuracy of the deep learning-based Chinese signature detection algorithm will be higher. The specific enhancement methods of the dataset enhancement module include:

[0107] Random erasing. Randomly select one or more regions in the image and fill them with the average value of nearby pixels. It is used to simulate the problem that key parts in the image may be occluded in real scenarios;

[0108] Random size. Randomly scale the size of the original image to [416 - 640], and then paste it into a blank image of 640×640 size;

[0109] Random cropping, random flipping;

[0110] Random angle rotation, where θ represents the rotation angle and the value range is θ ∈ (-5, 5);

[0111] Random brightness, contrast, and hue adjustment;

[0112] Random noise;

[0113] Random Gaussian blur;

[0114] Use the open source Mosaic enhancement method to randomly mix 4 pictures, which increases the number of batches in the training process and speeds up the training accordingly.

[0115] Step 103: training the Chinese signature detection algorithm model by using the enhanced data set. When the recognition accuracy of the Chinese signature detection algorithm model for the bounding box coordinates of the Chinese handwritten signature and the classification result reaches the threshold, the Chinese signature detection algorithm model is used as the final Chinese signature detection algorithm model.

[0116] Preferably, training the Chinese signature detection algorithm model by enhancing the data set also includes:

[0117] Based on the open source EfficientNet as the basic network, combined with the open source PANet, feature extraction of Chinese handwritten signature images is performed;

[0118] Extract the bounding box coordinates of the Chinese signature contained in the Chinese handwritten signature image through a detection algorithm;

[0119] The probability that the detected border coordinates are the border of the Chinese signature is calculated through a classification algorithm;

[0120] Based on the smooth L1 loss function, the border gap between the real Chinese signature border and the detected border coordinates is calculated to obtain the accuracy of the border gap;

[0121] The classification gap between the predicted classification result and the actual result is calculated based on the cross entropy loss function to obtain the classification gap accuracy;

[0122] When the border gap accuracy and the classification gap accuracy reach the threshold, the training of the Chinese signature detection algorithm model is stopped.

[0123] The present invention trains the Chinese signature detection algorithm model through a model training module. This part mainly designs a Chinese signature detection algorithm and uses the enhanced data set to continuously optimize the parameters of the training model so that it can achieve more accurate detection results. The present invention designs a new algorithm model based on deep learning and applied to Chinese handwritten signature detection. The specific model structure is shown in the attached figure. Figure 6 As shown. It mainly consists of three parts: feature extraction, detection classification and training. The specific description of each part is as follows:

[0124] 1) Feature extraction: It is used to extract features at different scales in the signature image. The open source EfficientNet is used as the basic network, and combined with the open source PANet to form a feature extraction structure.

[0125] 2) Detection and classification. It includes a detection branch and a classification branch. The detection branch is used to extract the rectangular boxes that may contain signatures in the signature images, and the classification branch is applied to predict the probability scores that the rectangular boxes are signatures. The algorithm structure diagram of the detection / classification part of the detection algorithm is as shown in the appendix Figure 6 as follows.

[0126] 3) Training. The smooth L1 loss function is used to calculate the gap between the true signature rectangular box and the predicted signature rectangular box, and the cross-entropy loss function is used to calculate the gap between the predicted classification result and the true result. The Adam optimizer is used to continuously update the model parameters. Finally, after multiple trainings, when the accuracy meets certain conditions, the training is stopped and the final model parameters are saved.

[0127] Step 104: Collect the Chinese handwritten signature images to be recognized, and preprocess the Chinese handwritten signature images to be recognized;

[0128] Preferably, collecting the Chinese handwritten signature images to be recognized and preprocessing the Chinese handwritten signature images to be recognized includes:

[0129] Collect the Chinese handwritten signature images to be recognized through a mobile terminal, a PC or an image acquisition device, or upload the Chinese handwritten signature images to be recognized from an album or a picture library; the color channels of the Chinese handwritten signature images to be recognized include color or grayscale, and the direction is positive upward;

[0130] Scale the width and height dimensions of the Chinese handwritten signature images to be recognized proportionally to 640×640, and normalize all pixel values to between 0 and 1 after dividing by 255.

[0131] The present invention uses the smooth L1 loss function to calculate the gap between the true signature rectangular box and the predicted signature rectangular box, uses the cross-entropy loss function to calculate the gap between the predicted classification result and the true result, and uses the Adam optimizer to continuously update the model parameters. Finally, after multiple trainings, when the accuracy meets certain conditions, the training is stopped and the final model parameters are saved.

[0132] The prediction stage mainly includes an image acquisition module, a Chinese handwritten signature detection module and an output module. As shown in the appendix Figure 8 as follows, which is the detailed flowchart of the prediction stage. The main purpose of the prediction stage is to use the model parameter file output in the training stage to perform accurate Chinese signature detection after collecting the images.

[0133] The specific functions of each module in the prediction stage are as follows:

[0134] The image acquisition module of the present invention is used to acquire pictures to be recognized. The acquisition of images includes, but is not limited to, taking pictures based on mobile and / or PC cameras, taking pictures with a high-definition instrument, uploading from an album, and uploading from a picture library, etc. The acquired image contains a Chinese handwritten signature image, as shown in the appendix Figure 9 , the color channel of the image is a color or grayscale image, and the direction is positive upward.

[0135] The present invention uses a Chinese handwritten signature detection module to identify the area of the Chinese signature in the picture. This part first loads the picture acquired by the image acquisition module and performs unified picture preprocessing operations, including proportionally scaling the width and height of the image to 640×640, dividing all pixel values of the image by 255, and normalizing all pixel values to between 0 and 1, etc. Then, the image data is sent into the model parameters trained in the training stage, and finally the result output by the model is obtained.

[0136] Step 105: Identify the preprocessed Chinese handwritten signature picture to be recognized through the final Chinese signature detection algorithm model, and output the border coordinates and classification results of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized.

[0137] Preferably, outputting the border coordinates and classification results of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized includes:

[0138] Output the border coordinates of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the coordinates of the center point of the border, Width is the width of the border, and Height is the height of the border;

[0139] Output the classification result of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized. The classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, retain the border coordinates of the Chinese handwritten signature picture to be recognized.

[0140] The present invention parses the result of the output part of the model through the output module. Through parsing, the detection box containing the Chinese signature in the final image and the probability value of this detection box are obtained. The specific flowchart is as shown in the appendix Figure 10As shown. The output of the model mainly includes two parts. One part is the detection result, which contains four coordinates Center_x / Center_y / Width / Height, representing the center point coordinates and width and height of the box respectively. The other part of the output is the classification result, which outputs the probability value corresponding to the detection box. Generally, a fixed threshold (such as 0.5) is set. When the score output by the classification is greater than this threshold, the detection box is retained; otherwise, it is filtered out. Finally, the coordinates of all detection boxes whose scores meet the requirements are output, which is the detection result of the signature in this image. The final detected signature box result is as attached Figure 11 as shown

[0141] The dataset generation module of the present invention generates a large number of handwritten signature pictures by using a large number of background libraries, personal name libraries, and font libraries, solves the problem of lack of datasets, and at the same time ensures the diversity of data;

[0142] The dataset enhancement module of the present invention completes various enhancement processes on the generated datasets, meets the requirements of pictures in various scenarios, enables the Chinese handwritten signature algorithm to achieve accurate detection performance in various scenarios, and enhances the robustness of the algorithm;

[0143] The feature extraction network in the Chinese handwritten signature detection algorithm of the present invention can better extract features at different scales of images with fewer model parameters, including shallow low-level features and deep high-level features, and has good feature expression ability;

[0144] The detection and classification module in the Chinese handwritten signature detection algorithm of the present invention separates the detection and classification branches, independently performs the detection and classification processes, and designs a simple and efficient convolutional layer to output the detection and classification results more quickly and accurately.

[0145] The Chinese handwritten signature detection method designed by the present invention obtains a large number of training datasets by using the method of generating datasets in the case of lack of handwritten signature data. At the same time, the method of dataset enhancement is used to make the generated datasets more suitable for various scenarios, making the data have better diversity. On this basis, the model structure of the feature extraction network and the detection and classification output branch is designed, which can complete the Chinese handwritten signature detection task more quickly and accurately, and at the same time reduces the number of model parameters, achieving the effect of lightweight.

[0146] Figure 12 It is a structural diagram of a Chinese handwritten signature recognition system based on deep learning according to a preferred embodiment of the present invention. As Figure 12 shown, the present invention provides a Chinese handwritten signature recognition system based on deep learning. The system includes:

[0147] A training unit 1201 is used to generate a training dataset including Chinese handwritten signature images; process the Chinese handwritten signature images to generate an enhanced dataset; train a Chinese signature detection algorithm model with the enhanced dataset. When the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification results of Chinese handwritten signatures respectively reach the thresholds, the Chinese signature detection algorithm model is used as the final Chinese signature detection algorithm model;

[0148] Preferably, the training unit 1201 is used to generate a training dataset including Chinese handwritten signature images, including:

[0149] Set the size of the image and the size of the font;

[0150] Randomly select a background from a background library, randomly select a name from a name library, and randomly select a font from a font library;

[0151] Use the background as the background of the image, add the name to the image according to the font and the size of the font, and generate a Chinese handwritten signature image.

[0152] Preferably, the training unit 1201 is used to process the Chinese handwritten signature images to generate an enhanced dataset, including at least one of the following processing methods:

[0153] Perform partial random erasure on the Chinese handwritten signature images;

[0154] Scale the Chinese handwritten signature images to a first preset size, and then paste them onto blank images of a second preset size; the second preset size is larger than the first preset size;

[0155] Perform random cropping or random flipping on the Chinese handwritten signature images;

[0156] Perform random angle rotation on the Chinese handwritten signature images, where θ represents the rotation angle, and the value range is θ ∈ (-5, 5);

[0157] Perform random brightness, contrast, and hue adjustment on the Chinese handwritten signature images;

[0158] Perform random noise adjustment on the Chinese handwritten signature images;

[0159] Perform random Gaussian blur adjustment on the Chinese handwritten signature images;

[0160] Randomly mix multiple Chinese handwritten signature images as new Chinese handwritten signature images through an open-source Mosaic enhancement method.

[0161] Preferably, the training unit 1201 is used to train the Chinese signature detection algorithm model with the enhanced dataset, and further includes:

[0162] Based on the open-source EfficientNet as the basic network and combined with the open-source PANet, feature extraction is performed on Chinese handwritten signature images;

[0163] The border coordinates of the Chinese signature contained in the Chinese handwritten signature image are extracted through a detection algorithm;

[0164] The probability that the detected border coordinates are the Chinese signature border is calculated through a classification algorithm;

[0165] Based on the smooth L1 loss function, the border gap between the true Chinese signature border and the detected border coordinates is calculated to obtain the border gap accuracy;

[0166] Based on the cross-entropy loss function, the classification gap between the predicted classification result and the true result is calculated to obtain the classification gap accuracy;

[0167] When the border gap accuracy and the classification gap accuracy reach the thresholds respectively, the training of the Chinese signature detection algorithm model is stopped.

[0168] The recognition unit 1202 is used to collect the Chinese handwritten signature image to be recognized and preprocess the Chinese handwritten signature image to be recognized; the preprocessed Chinese handwritten signature image to be recognized is recognized through the final Chinese signature detection algorithm model, and the border coordinates and classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized are output.

[0169] Preferably, the recognition unit 1202 is used to collect the Chinese handwritten signature image to be recognized and preprocess the Chinese handwritten signature image to be recognized, including:

[0170] The Chinese handwritten signature image to be recognized is collected through a mobile terminal, a PC or an image acquisition device, or the Chinese handwritten signature image to be recognized is uploaded from an album or a picture gallery; the color channel of the Chinese handwritten signature image to be recognized includes color or grayscale, and the direction is positive upward;

[0171] The width and height dimensions of the Chinese handwritten signature image to be recognized are scaled proportionally to 640×640, and all pixel values are normalized to between 0 and 1 after being divided by 255.

[0172] Preferably, the recognition unit is used to output the border coordinates and classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, including:

[0173] The border coordinates of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized are output, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the center point coordinates of the border, Width is the width of the border, and Height is the height of the border;

[0174] Output the classification result of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized. The classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, retain the border coordinates of the Chinese handwritten signature picture to be recognized.

[0175] A Chinese handwritten signature recognition system 1200 according to a preferred embodiment of the present invention corresponds to a Chinese handwritten signature recognition method 100 according to another preferred embodiment of the present invention, and will not be described in detail here.

[0176] The present invention has been described by referring to a few embodiments. However, as is well known to those skilled in the art, as defined by the appended patent claims, other embodiments equivalent to those disclosed above of the present invention equally fall within the scope of the present invention.

[0177] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless otherwise clearly defined therein. All references to "a / the [device, component, etc.]" are to be construed openly as at least one instance of the device, component, etc., unless otherwise explicitly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed, unless explicitly stated.

Claims

1. A Chinese handwritten signature recognition method based on deep learning, the method comprising: Generating a training data set including Chinese handwritten signature images; Processing the Chinese handwritten signature images to generate an enhanced data set; Training a Chinese signature detection algorithm model with the enhanced data set, and when the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification results of Chinese handwritten signatures respectively reach the thresholds, using the Chinese signature detection algorithm model as the final Chinese signature detection algorithm model; The training of the Chinese signature detection algorithm model with the enhanced data set further includes: Using the open-source EfficientNet as the basic network and combining the open-source PANet to extract features from the Chinese handwritten signature images; Extracting the border coordinates of the Chinese signature included in the Chinese handwritten signature images through a detection algorithm; Calculating the probability that the detected border coordinates are the borders of Chinese signatures through a classification algorithm; Calculating the border gap between the real Chinese signature border and the detected border coordinates based on the smooth L1 loss function to obtain the border gap accuracy; Calculating the classification gap between the predicted classification result and the real result based on the cross-entropy loss function to obtain the classification gap accuracy; When the border gap accuracy and the classification gap accuracy respectively reach the thresholds, stopping training the Chinese signature detection algorithm model; Collecting Chinese handwritten signature images to be recognized and preprocessing the Chinese handwritten signature images to be recognized; Recognizing the preprocessed Chinese handwritten signature images to be recognized through the final Chinese signature detection algorithm model, and outputting the border coordinates and classification results of the Chinese handwritten signatures in the Chinese handwritten signature images to be recognized.

2. The method according to claim 1, wherein the generating a training data set including Chinese handwritten signature images comprises: Setting the size of the image and the size of the font; Randomly selecting a background from a background library, randomly selecting a name from a name library, and randomly selecting a font from a font library; Using the background as the background of the image, adding the name to the image according to the font and the size of the font to generate a Chinese handwritten signature image.

3. The method according to claim 1, wherein the processing the Chinese handwritten signature images to generate an enhanced data set comprises at least one of the following processing methods: Performing partial random erasure on the Chinese handwritten signature images; Scaling the Chinese handwritten signature images to a first preset size and then pasting them onto blank images of a second preset size; The second preset size is larger than the first preset size; Performing random cropping or random flipping on the Chinese handwritten signature images; Performing random angle rotation on the Chinese handwritten signature images, where θ represents the rotation angle and the value range is θ ∈ (-5, 5); Performing random adjustment of brightness, contrast, and hue on the Chinese handwritten signature images; Performing random noise adjustment on the Chinese handwritten signature images; Performing random Gaussian blur adjustment on the Chinese handwritten signature images; Randomly mix multiple of the Chinese handwritten signature images through an open-source Mosaic enhancement method to obtain a new Chinese handwritten signature image.

4. The method according to claim 1, wherein the step of collecting the Chinese handwritten signature image to be recognized and preprocessing the Chinese handwritten signature image to be recognized includes: Collecting the Chinese handwritten signature image to be recognized through a mobile terminal, a PC or an image acquisition device, or uploading the Chinese handwritten signature image to be recognized from an album or a picture gallery; the color channel of the Chinese handwritten signature image to be recognized includes color or grayscale, and the direction is positive upward. Scaling the width and height dimensions of the Chinese handwritten signature image to be recognized proportionally to 640×640, and normalizing all pixel values to between 0 and 1 after dividing by 255.

5. The method according to claim 1, wherein the step of outputting the border coordinates and classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized includes: Outputting the border coordinates of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the coordinates of the center point of the border, Width is the width of the border, and Height is the height of the border. Outputting the classification result of the Chinese handwritten signature in the Chinese handwritten signature image to be recognized, where the classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, the border coordinates of the Chinese handwritten signature image to be recognized are retained.

6. A Chinese handwritten signature recognition system based on deep learning, the system includes: A training unit for generating a training data set including Chinese handwritten signature images; Processing the Chinese handwritten signature images to generate an enhanced data set; Training a Chinese signature detection algorithm model through the enhanced data set. When the recognition accuracies of the Chinese signature detection algorithm model for the border coordinates and classification result of the Chinese handwritten signature respectively reach the threshold, using the Chinese signature detection algorithm model as the final Chinese signature detection algorithm model. The training unit for training the Chinese signature detection algorithm model through the enhanced data set further includes: Using the open-source EfficientNet as the basic network and combining with the open-source PANet to extract features from the Chinese handwritten signature images; Extracting the border coordinates of the Chinese signature included in the Chinese handwritten signature images through a detection algorithm; Calculating the probability that the detected border coordinates are the borders of Chinese signatures through a classification algorithm; Calculating the border gap between the true Chinese signature border and the detected border coordinates based on the smooth L1 loss function to obtain the border gap accuracy; Calculating the classification gap between the predicted classification result and the true result based on the cross-entropy loss function to obtain the classification gap accuracy; When the border gap accuracy and the classification gap accuracy respectively reach the threshold, stopping training the Chinese signature detection algorithm model. An identification unit for collecting a Chinese handwritten signature picture to be identified and preprocessing the Chinese handwritten signature picture to be identified; identifying the preprocessed Chinese handwritten signature picture through a final Chinese signature detection algorithm model, and outputting the border coordinates and classification results of the Chinese handwritten signature in the Chinese handwritten signature picture to be identified.

7. The system according to claim 6, wherein the training unit is used to generate a training data set including Chinese handwritten signature pictures, including: Setting the size of the picture and the size of the font; Randomly selecting a background from a background library, randomly selecting a name from a name library, and randomly selecting a font from a font library; Using the background as the background of the picture, adding the name to the picture according to the font and the size of the font, and generating a Chinese handwritten signature picture.

8. The system according to claim 6, wherein the training unit is used to process the Chinese handwritten signature picture to generate an enhanced data set, including at least one of the following processing methods: Performing partial random erasure on the Chinese handwritten signature picture; Scaling the Chinese handwritten signature picture to a first preset size and then pasting it onto a blank picture of a second preset size; the second preset size is larger than the first preset size; Performing random cropping or random flipping on the Chinese handwritten signature picture; Performing random angle rotation on the Chinese handwritten signature picture, where θ represents the rotation angle and the value range is θ ∈ (-5, 5); Performing random adjustment of brightness, contrast, and hue on the Chinese handwritten signature picture; Performing random noise adjustment on the Chinese handwritten signature picture; Performing random Gaussian blur adjustment on the Chinese handwritten signature picture; Randomly mixing multiple Chinese handwritten signature pictures as a new Chinese handwritten signature picture through an open-source Mosaic enhancement method.

9. The system according to claim 6, wherein the identification unit is used to collect a Chinese handwritten signature picture to be identified and preprocess the Chinese handwritten signature picture to be identified, including: Collecting a Chinese handwritten signature picture to be identified through a mobile terminal, a PC, or an image acquisition device, or uploading a Chinese handwritten signature picture to be identified from an album or a gallery; the color channel of the Chinese handwritten signature picture to be identified includes color or grayscale, and the direction is positive upward; Scaling the width and height dimensions of the Chinese handwritten signature picture to be identified proportionally to 640×640, and normalizing all pixel values to between 0 and 1 after dividing by 255.

10. The system according to claim 6, wherein the identification unit is used to output the border coordinates and classification results of the Chinese handwritten signature in the Chinese handwritten signature picture to be identified, including: Outputting the border coordinates of the Chinese handwritten signature in the Chinese handwritten signature picture to be identified, including four coordinates Center_x / Center_y / Width / Height, where Center_x / Center_y are the coordinates of the center point of the border, Width is the width of the border, and Height is the height of the border; Output the classification result of the Chinese handwritten signature in the Chinese handwritten signature picture to be recognized. The classification result is the probability value that the border coordinates meet the recognition requirements. When the probability value is greater than the threshold, retain the border coordinates of the Chinese handwritten signature picture to be recognized.

Images