Mobile handwritten signature recognition method and device based on image processing and medium

Abstract

The embodiment of the invention discloses a mobile handwritten signature recognition method and device based on image processing and a medium, and relates to the technical field of handwritten signature recognition, and the method comprises the steps: obtaining the handwritten signature image information of a handwritten signature input in a mobile terminal, and carrying out the image analysis of the handwritten signature image information, determining a current dynamic threshold value corresponding to each pixel unit; based on the current dynamic threshold value corresponding to each pixel unit, binarization processing is carried out on the handwritten signature image information, a corresponding binarized signature image is determined to extract multi-dimensional signature features of the binarized signature image, and the multi-dimensional signature features comprise image features, dynamic features and geometric features; according to the multi-dimensional signature features and a pre-constructed signature recognition model, carrying out signature recognition on the handwritten signature, and determining recognition information of the handwritten signature; the method of dynamically adjusting the threshold value can significantly reduce errors in the binarization processing process, thereby improving the accuracy of subsequent signature recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of handwritten signature recognition, and in particular to a method, device and medium for mobile handwritten signature recognition based on image processing. Background Art

[0002] As an important branch of electronic signature technology, mobile handwritten signature approval technology has developed rapidly with the popularization of digital office. This mobile handwritten signature approval technology combines the portability of mobile devices with the legal effect of handwritten signatures, providing users with a more convenient and efficient way to sign. At present, mobile handwritten signature approval technology has realized the automation of the entire process from handwriting input, image preprocessing, feature extraction to signature recognition. Among them, the threshold binarization method, as one of the key steps in image preprocessing, is of great significance for improving the accuracy and stability of signature recognition.

[0003] Traditional threshold binarization methods usually use a fixed threshold for image segmentation, but handwritten signature images are often affected by multiple factors such as lighting and noise. When binarization is performed under a fixed threshold, the processing effect is not good. In addition, current signature recognition methods usually only rely on image features for recognition. However, handwritten signatures not only contain image features, but also dynamic features such as pressure and speed, as well as geometric features such as pen rise and fall and extreme points. Single feature recognition often cannot fully describe the characteristics of handwritten signatures, resulting in poor recognition results. Therefore, the traditional handwritten signature recognition process is limited by fixed thresholds and single features, resulting in signature recognition results that cannot meet accuracy requirements. Summary of the invention

[0004] One or more embodiments of the present specification provide a method, device and medium for mobile handwritten signature recognition based on image processing, which are used to solve the following technical problems: the traditional handwritten signature recognition process is limited by fixed thresholds and single features, resulting in the signature recognition effect failing to meet accuracy requirements.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of the present specification provide a mobile handwritten signature recognition method based on image processing, the method comprising: acquiring handwritten signature image information of a handwritten signature input in a mobile terminal, performing image analysis on the handwritten signature image information, and determining a current dynamic threshold corresponding to each pixel unit; binarizing the handwritten signature image information based on the current dynamic threshold corresponding to each pixel unit, determining a corresponding binarized signature image, and extracting multi-dimensional signature features of the binarized signature image, wherein the multi-dimensional signature features include image features, dynamic features, and geometric features; performing signature recognition on the handwritten signature according to the multi-dimensional signature features and a pre-built signature recognition model, and determining recognition information of the handwritten signature, wherein the recognition information includes any one of a true signature and a false signature.

[0007] One or more embodiments of this specification provide a mobile handwritten signature recognition device based on image processing, including:

[0008] at least one processor; and,

[0009] a memory communicatively connected to the at least one processor; wherein,

[0010] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0011] One or more embodiments of the present specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the above method.

[0012] At least one of the above-mentioned technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: through the above-mentioned technical solution, by determining the current dynamic threshold corresponding to each pixel unit, it is possible to more accurately adapt to different lighting conditions, ink depth and background noise in the handwritten signature image, and the method of dynamically adjusting the threshold can significantly reduce the error in the binarization process compared to the fixed threshold, thereby improving the accuracy of subsequent signature recognition; extracting multi-dimensional signature features of the binarized signature image, including image features, dynamic features and geometric features, provides rich information for signature recognition, more comprehensively reflects the uniqueness of the handwritten signature, and helps to distinguish between true signatures and false signatures; the determination of the dynamic threshold and the binarization process can effectively reduce noise interference in the handwritten signature image, such as background noise, trembling during handwriting, etc., thereby enhancing the signature recognition model. Its robustness to noise enables it to maintain stable recognition performance in different environments; multi-dimensional feature extraction can capture various subtle changes in handwritten signatures, including writing habits, style differences, etc., so that the signature recognition model can better adapt to the signature styles of different people, improving the versatility and accuracy of recognition; the dynamic threshold determination and binarization processing process is relatively simple and efficient, and can complete the processing of handwritten signature images in a short time, improve the real-time performance of signature recognition, and improve user experience; by matching the extracted multi-dimensional signature features with the pre-built signature recognition model, the authenticity of the handwritten signature can be verified in real time; because multi-dimensional feature extraction can capture subtle differences in handwritten signatures and has strong anti-counterfeiting capabilities, even if the forger can imitate the general shape of the signature, it is difficult to completely replicate the characteristics of the real signature in multiple dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0014] Figure 1 A flowchart of a mobile handwritten signature recognition method based on image processing provided in an embodiment of this specification;

[0015] Figure 2 A schematic diagram of the structure of a mobile handwritten signature recognition device based on image processing provided in an embodiment of this specification. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0017] The embodiment of this specification provides a mobile handwritten signature recognition method based on image processing. It should be noted that the execution subject in the embodiment of this specification can be a server or any device with data processing capabilities. Figure 1 A flowchart of a mobile handwritten signature recognition method based on image processing is provided in an embodiment of this specification, such as Figure 1 As shown, it mainly includes the following steps:

[0018] Step S101, obtaining the handwritten signature image information of the handwritten signature input in the mobile terminal, performing image analysis on the handwritten signature image information, and determining the current dynamic threshold corresponding to each pixel unit.

[0019] In one embodiment of the present specification, handwritten signature image information of a handwritten signature input by a user in a mobile terminal device is obtained. In addition to the handwritten signature image information, there are differences in the handwriting pressure, handwriting speed, etc. corresponding to the user when inputting the handwritten signature. The pressure data can be recorded by a sensor integrated in the mobile terminal device, and the signature generation time series data can be used to determine information such as the signature speed.

[0020] Performing image analysis on the handwritten signature image information to determine the current dynamic threshold corresponding to each pixel unit specifically includes: performing preprocessing operations on the handwritten signature image information to obtain an enhanced handwritten signature grayscale image, wherein the preprocessing operations include any one or more of grayscale processing, denoising processing and image enhancement processing; performing window division on the enhanced handwritten signature grayscale image according to a preset pixel unit division rule to determine multiple pixel units to extract pixel features of each pixel unit; and determining the current dynamic threshold corresponding to each pixel unit through the pixel features of each pixel unit.

[0021] In one embodiment of the present specification, first, the handwritten signature image is converted from color space to grayscale space. The grayscale image can simplify the subsequent processing steps while retaining sufficient signature information. The grayscale image is smoothed using a denoising algorithm such as a median filter and a Gaussian filter to reduce the impact of noise such as jitter during handwriting and paper texture on the binarization effect. If the contrast of the signature image is low, image enhancement techniques such as histogram equalization or contrast stretching can be used to improve the contrast of the image and make the signature clearer. According to the preset pixel unit division rule, the enhanced handwritten signature grayscale image is window-divided into multiple small windows (or local areas). The size of the window can be adjusted according to the stroke thickness and noise level of the signature. Generally, the window size should be larger than the minimum stroke width of the signature to ensure that the local features of the signature can be captured. For each pixel unit, pixel features are extracted, which may include grayscale value, mean, variance, gradient, etc. The grayscale value directly reflects the brightness information of the pixel; the mean and variance can reflect the brightness distribution within the pixel unit; the gradient can reflect the edge information within the pixel unit. According to the pixel characteristics of each pixel unit, the corresponding current dynamic threshold is determined.

[0022] Through the above technical scheme, through preprocessing operations (such as grayscale processing, denoising processing and image enhancement processing), the noise and interference in the image can be effectively reduced, making the features of the handwritten signature more prominent and clear, and the application of dynamic thresholds can more accurately distinguish the signature area and the background area, thereby improving the accuracy in the subsequent signature verification process; grayscale processing converts the color image into a grayscale image, reducing the influence of color information on signature recognition, so that the algorithm is more focused on the shape and texture features of the signature, denoising processing and image enhancement processing can further improve the image quality, so that the signature can maintain a good recognition effect under different lighting, angles and resolutions; the image is divided into multiple pixel units through window partitioning, and the pixel feature extraction and dynamic threshold calculation of each unit can be processed in parallel, thereby improving the overall processing speed; the dynamic threshold method avoids the errors that may be caused by the global fixed threshold, making the algorithm more flexible and efficient. In addition, the dynamic threshold method can determine the threshold according to the specific characteristics of each pixel unit, so it can adapt well to changes in different handwriting styles and signature conditions.

[0023] According to a preset pixel unit division rule, the enhanced handwritten signature grayscale image is divided into windows to determine a plurality of pixel units, specifically including: performing stroke feature extraction and noise feature extraction on the handwritten signature image information to obtain a plurality of stroke width features and a current image noise level index in the handwritten signature image information; determining a minimum reference threshold of a pixel window corresponding to the pixel unit according to a minimum stroke width among the plurality of stroke width features; determining a corresponding maximum reference threshold of a pixel window through a matching relationship between the current image noise level index and a preset noise window, wherein the noise window matching relationship includes a plurality of noise level indexes and a theoretical maximum value of a noise window corresponding to each noise level; determining a current pixel window size based on the minimum reference threshold of the pixel window and the maximum reference threshold of the pixel window, so as to perform window division on the enhanced handwritten signature grayscale image according to the current pixel window size to determine a plurality of pixel units.

[0024] In one embodiment of the present specification, image processing technology (such as edge detection, morphological processing, etc.) is used to extract stroke information in a handwritten signature image. By analyzing the morphology of the strokes, multiple stroke width features are calculated, and the stroke width features are used to reflect the thickness of different strokes in the signature. The enhanced handwritten signature grayscale image is subjected to noise analysis to calculate the noise level index of the current image. The noise level index here may include the proportion of noise pixels in the image, the mean and variance of the noise, etc. According to the extracted multiple stroke width features, the minimum stroke width is determined, and the minimum stroke width is used as the minimum reference threshold of the pixel window, ensuring that even the thinnest strokes can be completely captured in the window. The corresponding pixel window maximum reference threshold is determined using the current image noise level index and the preset noise window matching relationship. The preset noise window matching relationship can be a lookup table corresponding to the experimental process, and the corresponding theoretical noise window maximum value is given according to the noise level index. This maximum value is intended to ensure that the window is not too large, thereby avoiding the inclusion of too much noise or background information. Based on the pixel window minimum reference threshold and the pixel window maximum reference threshold, the size of the current pixel window is determined. The window size here can be the average of the two thresholds, or adjusted according to an empirical formula, but the window size should be a value between the minimum reference threshold of the pixel window and the maximum reference threshold of the pixel window. According to the determined current pixel window size, the enhanced handwritten signature grayscale image is divided into windows. Each window is a pixel unit for subsequent processing and analysis. By traversing the entire image, multiple pixel units can be determined.

[0025] Through the above technical scheme, the stroke information in the handwritten signature can be accurately identified through stroke feature extraction, including key features such as stroke width, which helps to determine the minimum reference threshold of the pixel window, thereby ensuring that important stroke details are not lost during the processing process. Considering the noise characteristics can make the processing more robust and avoid noise interference with signature recognition; the maximum reference threshold of the pixel window is determined by the current image noise level indicator and the preset noise window matching relationship, so that the size of the pixel window can be adaptive to different image qualities. In an image with large noise, a larger pixel window can be used to smooth the noise; and in an image with small noise, a smaller pixel window can be used to retain more detail information; the size of the pixel window is dynamically determined according to the stroke width and noise level of the handwritten signature, which can avoid the waste of resources that may be caused by using a fixed-size pixel window. In the case of thin strokes or low noise, using a smaller pixel window can reduce unnecessary calculations and improve processing efficiency; through fine pixel unit division, the characteristics of the handwritten signature can be more accurately captured, thereby improving the accuracy of signature recognition. Since the influence of noise is taken into account, it has higher stability when dealing with signature images with different noise levels.

[0026] The current dynamic threshold corresponding to each pixel unit is determined through the pixel characteristics of each pixel unit, specifically including: obtaining the pixel characteristics of each pixel unit, wherein the pixel characteristics include the number of pixels of each gray level in the pixel unit; pre-setting multiple gray thresholds, dividing the pixel unit based on each gray threshold, and determining the current foreground area and the current background area corresponding to each pixel unit; calculating the current distinguishing index corresponding to each gray threshold through the current foreground area and the current background area corresponding to each pixel unit, and screening among the multiple gray thresholds according to the multiple current distinguishing indexes to determine each corresponding current dynamic threshold.

[0027] In one embodiment of the present specification, the number of pixels at each gray level (0-255) is counted for each pixel unit to form a gray histogram, in which each gray level corresponds to a number of pixels. According to the characteristics of the handwritten signature and the overall gray distribution of the image, a series of possible gray thresholds are pre-set to divide the pixel unit into foreground and background areas. For each pixel unit and each preset gray threshold, the pixels in the pixel unit are divided into two categories, including pixels with gray values ​​less than or equal to the threshold as the current foreground area, and pixels with gray values ​​greater than the threshold as the current background area. For each gray threshold and each pixel unit, a distinction index is calculated to evaluate the performance of the threshold in distinguishing the foreground and background areas. Commonly used distinction indexes include inter-class variance (such as variance in the Otsu method), average gray difference between foreground and background areas, ratio of the number of pixels in foreground and background areas, etc. According to the calculated multiple current distinction indexes, screening is performed among multiple preset gray thresholds. The gray threshold with the best distinction index (such as the maximum inter-class variance, the maximum average gray difference, etc.) is selected as the current dynamic threshold of the current pixel unit.

[0028] Through the above technical scheme, by obtaining the pixel features of each pixel unit (i.e., the number of pixels of each gray level in the pixel unit), the grayscale distribution of the image can be understood in more detail; by pre-setting multiple grayscale thresholds and dividing each pixel unit, it is possible to adapt to the grayscale changes of different images and scenes, which is more flexible than the fixed threshold method and can better process complex images; by dividing the pixel units based on the grayscale threshold, the current foreground area and the current background area can be determined; by calculating the current distinction index corresponding to each grayscale threshold (such as inter-class variance, information entropy, etc.), the ability of different thresholds to distinguish foreground and background areas can be quantified, and quantitative evaluation helps to screen out the optimal current dynamic threshold from multiple grayscale thresholds; by screening among multiple grayscale thresholds according to multiple current distinction indexes to determine the current dynamic threshold corresponding to each pixel unit, the threshold can be adaptively adjusted to adapt to the grayscale changes in different image areas, thereby improving the accuracy and robustness of image processing.

[0029] Step S102, based on the current dynamic threshold corresponding to each pixel unit, the handwritten signature image information is binarized to determine the corresponding binarized signature image to extract the multi-dimensional signature features of the binarized signature image.

[0030] The multi-dimensional signature features include image features, dynamic features and geometric features;

[0031] Based on the current dynamic threshold corresponding to each pixel unit, the handwritten signature image information is binarized to determine the corresponding binarized signature image, specifically including: obtaining the enhanced handwritten signature grayscale image corresponding to the handwritten signature image information, dividing the enhanced handwritten signature grayscale image according to each pixel unit, and determining the corresponding grayscale sub-region; determining the current grayscale value of each pixel point in each grayscale sub-region, and determining the binarized sub-region image corresponding to each grayscale sub-region based on the current grayscale value and the current dynamic threshold; determining the corresponding binarized signature image through the binarized sub-region image corresponding to each grayscale sub-region.

[0032] In one embodiment of the present specification, the enhanced handwritten signature grayscale image is divided according to each pixel unit, and the corresponding grayscale sub-region is determined, and the image is divided into smaller and more manageable regions so that each part can be binarized more accurately. The current grayscale value of each pixel in each grayscale sub-region is determined, and each grayscale sub-region is binarized based on the current dynamic threshold corresponding to each pixel unit. Specifically, the pixel points with grayscale values ​​higher than the dynamic threshold are set to 1, and the pixel points with grayscale values ​​lower than or equal to the dynamic threshold are set to 0. After the above processing, each grayscale sub-region will correspond to a binary sub-region image. These sub-region images remove redundant grayscale information while retaining the key features of the signature, making the image more concise and clear. The binary sub-region images corresponding to each grayscale sub-region are combined to form a complete binary signature image.

[0033] Through the above technical solution, by setting the current dynamic threshold for each pixel unit, it is possible to more accurately adapt to the grayscale changes in different areas of the handwritten signature image, avoiding the over-segmentation or under-segmentation problem that may be caused by the traditional fixed threshold method; the enhanced handwritten signature grayscale image is divided into multiple grayscale sub-regions, which can further refine the processing process, and each sub-region can be binarized according to its specific grayscale distribution and characteristics, thereby improving the overall binarization accuracy; dynamic threshold binarization can better retain the key features of the handwritten signature, such as the thickness, shape, texture, etc. of the strokes; through accurate binarization processing, the noise interference in the image, such as background noise, uneven ink, etc., can be effectively reduced, and the accuracy and robustness of signature recognition can be improved; the dynamic threshold binarization method simplifies the complex processes in traditional binarization processing, such as threshold selection, image segmentation, etc., reduces processing time, improves overall efficiency, and greatly reduces the amount of image data after binarization, reduces storage requirements, and is suitable for application scenarios of large-scale signature data; accurate binarization processing helps to improve the accuracy of signature recognition, and by retaining more signature features and reducing noise interference, the recognition model can more accurately recognize the handwritten signature.

[0034] Extracting multi-dimensional signature features of the binary signature image specifically includes: performing signature contour extraction and texture feature extraction on the binary signature image to determine the shape features and texture features corresponding to the handwritten signature, and determining the image features corresponding to the handwritten signature based on the shape features and the texture features; obtaining, through the mobile terminal, signature pressure features and signature speed features corresponding to the handwritten signature image information to determine the dynamic features corresponding to the handwritten signature; performing pen-up and pen-down feature extraction and curvature extreme point feature extraction on the binary signature image to determine the pen-up and pen-down features and extreme point features corresponding to the handwritten signature to determine the geometric features corresponding to the handwritten signature; determining the multi-dimensional signature features of the binary signature image through the image features, the dynamic features and the geometric features corresponding to the handwritten signature.

[0035] In one embodiment of the present specification, a contour detection algorithm (such as Canny edge detection) is used to extract the contour of a handwritten signature. Afterwards, the shape features can be described by calculating the perimeter, area, rectangularity (the ratio of the area of ​​the minimum circumscribed rectangle of the signature area to the area of ​​the signature area), circularity (the ratio of the perimeter of the signature area to the perimeter of a circle of the same area) and other parameters of the contour. Based on the idea of ​​texture analysis, frequency domain analysis methods (such as Fourier transform and wavelet transform) can be used to extract the spectral features of the image as texture features. In addition, methods such as local binary patterns or gray-level co-occurrence matrices can also be used to extract texture features. Since the binarized image does not contain pressure information, it is impossible to directly extract pressure features from the image. However, if the handwritten signature is completed on a device with pressure sensing function, the pressure data can be recorded by the sensor of the device and extracted as a feature. Similarly, since the binarized image does not contain speed information, it is impossible to directly extract speed features from the image. However, the time series data of the handwriting is recorded during the signature process, and the speed features can be estimated by calculating the time difference and distance difference between adjacent handwriting points. By analyzing the starting and ending points of the handwriting, the starting and ending positions of the signature can be determined. In a binary image, the starting and ending points are usually represented by the starting and ending pixels of the handwriting. The extreme points can be the local maximum or minimum curvature points, direction change points, or speed change points in the handwriting. In a binary image, the extreme points can be detected by calculating the gradient or curvature of the pixel points. In addition, algorithms such as scale-invariant feature transforms or accelerated robust features can be used to detect key points in the image, which usually correspond to extreme points in the handwriting. First, the shape features are extracted using a contour detection algorithm; then, the texture features are extracted using methods such as frequency domain analysis; based on the pressure and time series data recorded during the signature process, the pressure and speed features are extracted; finally, the feature set is improved by analyzing the geometric features such as the starting point, end point, and extreme points of the handwriting.

[0036] Step S103, performing signature recognition on the handwritten signature according to the multi-dimensional signature features and the pre-built signature recognition model, and determining recognition information of the handwritten signature.

[0037] The identification information includes any one of a true signature and a false signature.

[0038] According to the multi-dimensional signature feature and the pre-constructed signature recognition model, the handwritten signature is recognized, and before the recognition information of the handwritten signature is determined, the method also includes: constructing an initial signature recognition model, wherein the initial signature recognition model includes a convolution layer, a pooling layer, a fully connected layer and an output layer; inputting the signature image and the signature label in the pre-acquired signature image data set into the initial signature recognition model, updating the weight parameters and bias parameters of the initial signature recognition model according to the cross entropy loss function and the Adam optimization algorithm, and determining the signature recognition model.

[0039] In one embodiment of the present specification, an initial signature recognition model is constructed, wherein the initial signature recognition model includes a convolution layer, a pooling layer, a fully connected layer, and an output layer. The convolution layer is the core part of the convolutional neural network. The input image is feature extracted through a convolution operation. Multiple convolution layers are designed, and each convolution layer uses a different convolution kernel size and number to extract features at different levels. The pooling layer is used to reduce the resolution of the image, reduce the number of parameters, and improve the robustness of the model. The fully connected layer maps the features of the convolution and pooling layers to the output space to achieve the classification task, design one or more fully connected layers, and use the activation function Sigmoid for nonlinear transformation. The output layer usually uses the Softmax function for multi-classification tasks and outputs the probability of each category. In signature recognition, the output layer will output the probability of true signatures and false signatures. The cross entropy loss function is selected to measure the difference between the model prediction result and the true label. The Adam optimization algorithm is selected to update the weights and biases of the model to minimize the loss function. The preprocessed signature image and the corresponding label are input into the model, and the forward propagation is performed to calculate the loss. Update the model's weights and biases based on the loss function and optimization algorithm.

[0040] Collect a large amount of image data of known signatures, which should cover signatures with different writing styles, paper quality, and ink conditions. Preprocess the collected signature images, including denoising, contrast enhancement, binarization, etc., to extract clear signature features. Use image processing technology and machine learning algorithms to extract multi-dimensional features from the preprocessed signature images. Use the extracted features as input and the signature category as output to train the signature recognition model so that the model can learn the mapping relationship between signature features and signature categories.

[0041] In one embodiment of the present specification, the evaluation index may select the F1 score to evaluate the performance of the model. The model is optimized according to the evaluation results, including adjusting the parameters of the convolution layer, the pooling layer, and the fully connected layer, using data enhancement technology, regularization technology, etc. The signature image to be recognized is input into the trained model, and the forward propagation calculation is performed to obtain the probability of the true signature and the false signature. The authenticity of the signature is judged according to the probability value.

[0042] In one embodiment of the present specification, the multi-dimensional signature features are input into a pre-built signature recognition model, signature recognition is performed on the handwritten signature, and probability data corresponding to the true signature and the false signature are output, so as to determine the recognition information of the handwritten signature based on the probability data corresponding to the true signature and the false signature. Through the above technical solution, in addition to the basic image features, the dynamic features such as pressure and speed of the handwritten signature, as well as the geometric features such as the starting and ending strokes and extreme points of the signature strokes are also integrated to form a signature recognition model with multiple features fusion. This multi-feature fusion method can more comprehensively describe the features of the handwritten signature and improve the accuracy and robustness of recognition.

[0043] In actual application scenarios of mobile handwritten signatures, there are problems such as diverse signature styles, different writing speeds, and blurred ink, which lead to differences in the presentation length of handwritten signatures. Longer signature types also increase the difficulty of signature recognition.

[0044] After obtaining the handwritten signature image information of the handwritten signature input in the mobile terminal, the method also includes: performing signature length recognition on the handwritten signature image information to determine the current signature length data corresponding to the handwritten signature; obtaining multiple verification signature images in a pre-constructed verification data subset and a signature label corresponding to each of the verification signature images, so as to segment the verification signature image according to multiple reference segment lengths, and determine a verification signature image set corresponding to each reference segment length; through the signature recognition model, identifying each segmented verification signature in the verification signature image set, determining the model output information corresponding to each reference segment length, and determining the current segment length corresponding to the signature recognition model based on the model output information corresponding to each reference segment length; segmenting the handwritten signature based on the current signature length data and the current segment length, and determining multiple signature segment image information.

[0045] In one embodiment of the present specification, a handwritten signature image to be processed is obtained. The handwritten signature image is preprocessed by grayscale, binarization, denoising and other preprocessing operations to improve the accuracy of subsequent processing. The current signature length data of the handwritten signature is determined using image processing technology (such as contour detection, pixel counting and other technologies), where the current signature length data can be the pixel length of the signature in the image or the character length. Multiple verification signature images of known signature labels are obtained, including multiple signature lengths, to ensure that each verification signature image has a corresponding signature label, which may include signature authenticity information.

[0046] According to a plurality of preset reference segment lengths (such as short, medium, long, etc., corresponding to one character, two characters and three characters respectively), each verification signature image is segmented, which can be achieved by dividing the image into sub-areas of different lengths in the horizontal direction. For each reference segment length, a set of verification signature images segmented under this length is created. For example, the verification signature image corresponding to ABC includes the character A image, character B image, character C image under the reference segment length of one character, and can also include the character AB image and character C image under two characters, and can also include the character ABC image under three characters. The segmented verification signature in each verification signature image set is recognized by the signature recognition model, and the classification probability in the model output information is recorded. Based on the model output information, the reference segment length with the best performance is selected as the current segment length, indicating that the model can recognize the signature most accurately under this length.

[0047] The handwritten signature is segmented using the determined current signature length data and the current segment length. The handwritten signature image is divided into multiple signature segment image information, each of which contains a part of the signature. In the above steps, the segmented signatures are recognized in turn, and the longer handwritten signature is divided into several segments for processing. Each segment of the signature can be binarized, feature extracted, and signature recognized, which can reduce the processing complexity and improve the recognition accuracy.

[0048] Through the above technical scheme, by determining the current dynamic threshold corresponding to each pixel unit, it is possible to more accurately adapt to different lighting conditions, ink depth and background noise in the handwritten signature image. The method of dynamically adjusting the threshold can significantly reduce the error in the binarization process compared to the fixed threshold, thereby improving the accuracy of subsequent signature recognition; extracting multi-dimensional signature features of the binarized signature image, including image features, dynamic features and geometric features, provides rich information for signature recognition, more comprehensively reflects the uniqueness of the handwritten signature, and helps to distinguish between true signatures and false signatures; the determination of the dynamic threshold and the binarization process can effectively reduce noise interference in the handwritten signature image, such as background noise, trembling during handwriting, etc., enhance the robustness of the signature recognition model to noise, and make it more effective in different environments. It can maintain stable recognition performance under all circumstances; multi-dimensional feature extraction can capture various subtle changes in handwritten signatures, including writing habits, style differences, etc., so that the signature recognition model can better adapt to the signature styles of different people, improving the versatility and accuracy of recognition; the dynamic threshold determination and binarization processing process is relatively simple and efficient, and can complete the processing of handwritten signature images in a short time, improve the real-time performance of signature recognition, and improve user experience; by matching the extracted multi-dimensional signature features with the pre-built signature recognition model, the authenticity of the handwritten signature can be verified in real time; because multi-dimensional feature extraction can capture subtle differences in handwritten signatures and has strong anti-counterfeiting capabilities, even if the forger can imitate the general shape of the signature, it is difficult to completely replicate the characteristics of the real signature in multiple dimensions.

[0049] The embodiment of this specification also provides a mobile handwritten signature recognition device based on image processing, such as Figure 2 As shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method.

[0050] The embodiments of the present specification also provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the above method.

[0051] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0052] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0053] The devices and media provided in the embodiments of this specification correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0054] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0055] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0056] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0057] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0058] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0059] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0060] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0061] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0062] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A mobile handwritten signature recognition method based on image processing, characterized in that: The method comprises: Acquire handwritten signature image information of a handwritten signature input in a mobile terminal, perform image analysis on the handwritten signature image information, and determine a current dynamic threshold corresponding to each pixel unit; Based on the current dynamic threshold corresponding to each pixel unit, the handwritten signature image information is binarized to determine the corresponding binarized signature image, so as to extract the multi-dimensional signature feature of the binarized signature image, wherein the multi-dimensional signature feature includes image feature, dynamic feature and geometric feature; The handwritten signature is recognized based on the multi-dimensional signature feature and a pre-built signature recognition model to determine recognition information of the handwritten signature, wherein the recognition information includes any one of a true signature and a false signature.

2. According to the method of mobile handwritten signature recognition based on image processing in claim 1, it is characterized in that: Performing image analysis on the handwritten signature image information to determine the current dynamic threshold corresponding to each pixel unit specifically includes: Performing a preprocessing operation on the handwritten signature image information to obtain an enhanced handwritten signature grayscale image, wherein the preprocessing operation includes any one or more of grayscale processing, denoising processing, and image enhancement processing; According to a preset pixel unit division rule, the enhanced handwritten signature grayscale image is divided into windows to determine a plurality of pixel units to extract pixel features of each of the pixel units; The current dynamic threshold corresponding to each pixel unit is determined according to the pixel feature of each pixel unit.

3. The mobile handwritten signature recognition method based on image processing according to claim 2 is characterized in that: According to a preset pixel unit division rule, the enhanced handwritten signature grayscale image is divided into windows to determine a plurality of pixel units, specifically including: Performing stroke feature extraction and noise feature extraction on the handwritten signature image information to obtain a plurality of stroke width features and a current image noise level index in the handwritten signature image information; Determine a minimum reference threshold of a pixel window corresponding to the pixel unit according to a minimum stroke width among the plurality of stroke width features; Determine the corresponding pixel window maximum reference threshold through the current image noise level index and the preset noise window matching relationship, wherein the noise window matching relationship includes multiple noise level indexes and the theoretical noise window maximum value corresponding to each noise level; Based on the pixel window minimum reference threshold and the pixel window maximum reference threshold, the current pixel window size is determined, so as to divide the enhanced handwritten signature grayscale image into windows according to the current pixel window size to determine a plurality of pixel units.

4. The mobile handwritten signature recognition method based on image processing according to claim 3 is characterized in that: Determining a current dynamic threshold corresponding to each pixel unit according to the pixel feature of each pixel unit specifically includes: Acquire a pixel feature of each of the pixel units, wherein the pixel feature includes the number of pixels of each gray level in the pixel unit; Pre-setting a plurality of grayscale thresholds, dividing the pixel units based on each of the grayscale thresholds, and determining a current foreground area and a current background area corresponding to each of the pixel units; The current distinguishing index corresponding to each grayscale threshold is calculated through the current foreground area and the current background area corresponding to each pixel unit, and the multiple grayscale thresholds are screened according to the multiple current distinguishing indexes to determine each corresponding current dynamic threshold.

5. The mobile handwritten signature recognition method based on image processing according to claim 1 is characterized in that: Based on the current dynamic threshold corresponding to each pixel unit, the handwritten signature image information is binarized to determine the corresponding binarized signature image, specifically including: Acquire an enhanced handwritten signature grayscale image corresponding to the handwritten signature image information, and divide the enhanced handwritten signature grayscale image according to each of the pixel units to determine a corresponding grayscale sub-region; Determine a current grayscale value of each pixel in each grayscale sub-region, and determine a binary sub-region image corresponding to each grayscale sub-region based on the current grayscale value and the current dynamic threshold; The corresponding binary signature image is determined through the binary sub-region image corresponding to each of the grayscale sub-regions.

6. The mobile handwritten signature recognition method based on image processing according to claim 1 is characterized in that: Extracting the multi-dimensional signature features of the binary signature image specifically includes: Performing signature contour extraction and texture feature extraction on the binary signature image to determine shape features and texture features corresponding to the handwritten signature, and determining image features corresponding to the handwritten signature based on the shape features and the texture features; Acquiring, by means of the mobile terminal, signature pressure characteristics and signature speed characteristics corresponding to the handwritten signature image information, so as to determine dynamic characteristics corresponding to the handwritten signature; Performing pen-starting and pen-ending feature extraction and curvature extreme point feature extraction on the binary signature image to determine the pen-starting and pen-ending features and extreme point features corresponding to the handwritten signature, so as to determine the geometric features corresponding to the handwritten signature; The multi-dimensional signature features of the binary signature image are determined by the image features, the dynamic features and the geometric features corresponding to the handwritten signature.

7. The method for mobile handwritten signature recognition based on image processing according to claim 1, characterized in that: Before performing signature recognition on the handwritten signature according to the multi-dimensional signature feature and the pre-built signature recognition model and determining the recognition information of the handwritten signature, the method further includes: Constructing an initial signature recognition model, wherein the initial signature recognition model includes a convolutional layer, a pooling layer, a fully connected layer, and an output layer; The signature image and signature label in the pre-acquired signature image data set are input into the initial signature recognition model, and the weight parameters and bias parameters of the initial signature recognition model are updated according to the cross entropy loss function and the Adam optimization algorithm to determine the signature recognition model.

8. The mobile handwritten signature recognition method based on image processing according to claim 1 is characterized in that: After acquiring the handwritten signature image information of the handwritten signature input in the mobile terminal, the method further includes: Performing signature length recognition on the handwritten signature image information to determine current signature length data corresponding to the handwritten signature; Acquire multiple verification signature images and signature labels corresponding to each verification signature image in a pre-constructed verification data subset, segment the verification signature images according to multiple reference segment lengths, and determine a verification signature image set corresponding to each reference segment length; Recognize the segmented verification signature in each of the verification signature image sets through the signature recognition model, determine the model output information corresponding to each of the reference segment lengths, and determine the current segment length corresponding to the signature recognition model based on the model output information corresponding to each of the reference segment lengths; Based on the current signature length data and the current segment length, the handwritten signature is segmented to determine a plurality of signature segment image information.

9. A mobile handwritten signature recognition device based on image processing, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to execute the method according to any one of claims 1 to 8.

Images