A recognition-based electronic signature handwriting segmentation method

Abstract

The application relates to a recognition-based electronic signature handwriting segmentation method, which comprises the following steps: constructing and training a handwriting segmentation model in advance; obtaining an image to be processed; inputting the image to be processed into the handwriting segmentation model; the handwriting segmentation model outputs a prediction result, the prediction result comprising a plurality of boundary boxes and a plurality of target confidence degrees; and the plurality of boundary boxes are combined to obtain handwriting segmentation results.

Description

Technical Field

[0001] This invention relates to a recognition-based method for handwritten electronic signature segmentation, belonging to the field of handwriting segmentation. Background Technology

[0002] Handwriting segmentation refers to the process of decomposing handwritten strokes from writing trajectory information acquired by data acquisition devices (such as touch screens, digital writing tablets, etc.) into isolated Chinese characters. The results of handwriting segmentation have a significant impact on the accuracy of subsequent Chinese character recognition. However, due to the great randomness of handwritten Chinese characters, the complex and diverse positional relationships between adjacent characters, and the tendency for strokes to stick together and intersect, there is still a lack of a method that can accurately segment handwritten strokes in electronic signatures.

[0003] Common Chinese character segmentation methods include stroke-based methods, pixel-tracking-based methods, and neural network-based recognition methods. Compared to other methods, recognition-based Chinese character segmentation methods, while dynamically selecting segmentation points and reducing segmentation errors, are time-consuming per character and still suffer from significant instances of duplicate and missed detections. Figure 2 As shown, different components of the same Chinese character are separated independently, or they are merged and separated together with other Chinese characters.

[0004] The YOLOv1 network structure consists of 24 convolutional layers, 1 unfolding layer (which unfolds the multi-dimensional convolutional results into a one-dimensional structure, serving as a transition), and 2 fully connected layers. The object detection steps are as follows: First, the input image is scaled to a fixed size of 448×448. Then, the image is input into the convolutional neural network, which outputs the coordinates of the predicted bounding boxes, the class of the object within the box, and the confidence score (i.e., outputting a 7×7×30 tensor, where 30 = (20 class probabilities + 2×5 (confidence score, bounding box position)). YOLOv1's advantage is its very fast detection speed. However, due to the relatively coarse image grid division, the number of bounding boxes generated per grid is small, resulting in poor detection performance for small and nearby objects, and a relatively large localization error, leading to lower overall detection accuracy. Summary of the Invention

[0005] To overcome the problems existing in the prior art, this invention designs a handwritten electronic signature segmentation method based on recognition. By merging bounding boxes, it solves the problem of single characters being segmented into multiple characters due to broken strokes and character structure, thereby improving the accuracy of handwriting segmentation.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Technical Solution 1:

[0008] A handwritten signature segmentation method based on recognition includes the following steps:

[0009] Pre-build and train a handwriting segmentation model;

[0010] Acquire the image to be processed; input the image to be processed into the handwriting segmentation model, and the handwriting segmentation model outputs the prediction results, which include several bounding boxes and several target confidence scores;

[0011] The several bounding boxes are merged to obtain the handwriting segmentation result.

[0012] Furthermore, the handwriting segmentation model includes an input layer, a backbone network, a first convolutional layer, a second convolutional layer, and an output layer.

[0013] Furthermore, the kernel of the first convolutional layer is 3*3; the kernel of the second convolutional layer is 1*1.

[0014] Furthermore, the backbone network includes several convolutional layers and several residual structures.

[0015] Furthermore, merge the bounding boxes that coincide in the Y-axis direction.

[0016] Furthermore, merge bounding boxes whose spacing along the X-axis is less than the spacing threshold and whose variance of the width of each bounding box is reduced after merging.

[0017] Furthermore, the spacing threshold is determined based on the average width of each bounding box.

[0018] Furthermore, the handwriting segmentation model is iteratively trained using a loss function, where the loss value calculated by the loss function is the sum of the target prediction loss value and the bounding box prediction loss value.

[0019] Further, the handwriting coordinate information of the user's electronic signature is obtained; a handwriting image is generated based on the handwriting coordinate information, and the handwriting image is input into the handwriting segmentation model as an image to be processed.

[0020] Technical Solution Two:

[0021] A handwriting segmentation device based on electronic signature recognition includes a processor and a memory for storing executable instructions of the processor and a handwriting segmentation model; the processor is configured to execute the following instructions:

[0022] Obtain the image to be processed;

[0023] Input the image to be processed into the handwriting segmentation model, and the handwriting segmentation model outputs the prediction results, which include several bounding boxes and several target confidence scores;

[0024] The several bounding boxes are merged to obtain the handwriting segmentation result.

[0025] Furthermore, the handwriting segmentation model includes an input layer, a backbone network, a first convolutional layer, a second convolutional layer, and an output layer.

[0026] Furthermore, the kernel of the first convolutional layer is 3*3; the kernel of the second convolutional layer is 1*1.

[0027] Furthermore, the backbone network includes several convolutional layers and several residual structures.

[0028] Furthermore, merge the bounding boxes that coincide in the Y-axis direction.

[0029] Furthermore, merge bounding boxes whose spacing along the X-axis is less than the spacing threshold and whose variance of the width of each bounding box is reduced after merging.

[0030] Furthermore, the spacing threshold is determined based on the average width of each bounding box.

[0031] Furthermore, the handwriting segmentation model is iteratively trained using a loss function, where the loss value calculated by the loss function is the sum of the target prediction loss value and the bounding box prediction loss value.

[0032] Further, the handwriting coordinate information of the user's electronic signature is obtained; a handwriting image is generated based on the handwriting coordinate information, and the handwriting image is input into the handwriting segmentation model as an image to be processed.

[0033] Compared with the prior art, the present invention has the following features and beneficial effects:

[0034] 1. This invention modifies the backbone network in YOLOv1 to ResNet18, and modifies the flattening layer and fully connected layer to 3*3 and 1*1 convolutional layers. The modified YOLOv1 is used as the handwriting segmentation model. ResNet18 can effectively improve the feature extraction speed and bounding box location accuracy of the handwriting segmentation model. The first and second convolutional layers add more spatial information, enabling the handwriting segmentation model to learn more features, thereby further improving the bounding box location accuracy obtained by the handwriting segmentation model.

[0035] 2. In this invention, the loss function in YOLOv is modified. After modification, the loss function L only calculates the target prediction loss value and the bounding box prediction loss value. Therefore, the handwriting segmentation model trained only outputs the target feature 14*14*1 and the bounding box feature 14*14*4. The backbone network does not need to extract features of the target category. The training amount of the handwriting segmentation model is greatly reduced and the calculation speed is improved.

[0036] 3. This invention improves the accuracy of handwriting segmentation by merging bounding boxes, which solves the problem of single characters being segmented into multiple characters due to broken strokes and character structure.

[0037] 4. Since electronic signatures contain only a small amount of text (only 2 to 4 characters), the average value of the bounding box width fluctuates greatly before and after text merging. This invention uses the variance value of the bounding box width to further improve the accuracy of handwriting segmentation. This method has low computational cost and fast segmentation speed. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the handwriting segmentation model of the present invention;

[0039] Figure 2 This is a schematic diagram of the handwriting segmentation results using existing technology;

[0040] Figure 3 , 4 This is a schematic diagram of the handwriting segmentation results of the present invention. Detailed Implementation

[0041] The present invention will now be described in more detail with reference to the embodiments.

[0042] Example 1

[0043] A handwritten signature segmentation method based on recognition includes the following steps:

[0044] S1, such as Figure 1 As shown, a handwriting segmentation model is constructed: The handwriting segmentation model includes an input layer, a backbone network (in this embodiment, the backbone network is ResNet18, which includes 17 convolutional layers, 1 fully connected hidden layer, and multiple residual structures), a first convolutional layer (size 3*3*512), a second convolutional layer (size 1*1*5), and an output layer.

[0045] S2. Construct the loss function L = L obj +L bbox ;

[0046] Among them, L ob j is the target prediction loss function; L bbox The loss function for bounding box prediction.

[0047]

[0048]

[0049] In the formula, λ obj =5; λ noobj =1; s 2 This represents the length and width dimensions of the features extracted by the backbone network. In this embodiment, s 2 =14*14; The label value represents the input data; and As the control function, when the i-th grid point contains the target, For 1, When the value is 0, the i-th grid point does not contain the target. 0, =1; λ bbox =5; x i and These represent the actual and predicted positions of the bounding box along the x-axis, respectively, and the y-axis... i and w represents the actual and predicted positions of the bounding box along the y-axis, respectively. i and h represents the actual width and predicted width of the bounding box, respectively. i and These represent the actual height and the predicted height of the bounding box, respectively.

[0050] S3. Construct the training dataset:

[0051] Read the handwriting coordinate file from the electronic signature device; create a new signature trajectory layer, and use the coordinate information in the handwriting coordinate file to reconstruct the signature trajectory. On the layer, a pixel value of 0 represents the background and a pixel value of 255 represents the signature trajectory, generating a handwriting image.

[0052] Label the coordinates of each text region in the handwriting image, in the format (x, y, w, h, c), where x represents the x-axis position of the text box, y represents the y-axis position, w represents the width of the text box, h represents the height of the text box, and c indicates whether the region is a text area. Use multiple labeled handwriting images as the training dataset, and divide them into training, test, and validation sets in a ratio of 8:1:1.

[0053] S4. Set hyperparameters and train the handwriting segmentation model.

[0054] The initial learning rate was set to 0.1, the batch size of training data was 64, and the number of training iterations was 10 epochs. The training set was input into the handwriting segmentation model, and the model was trained iteratively using the loss function L. During training, the accuracy of the model was tested using the test set data to observe whether there were significant fluctuations in the model's accuracy and to verify the model's generalization ability. The model's accuracy was tested using the validation set, and the model's accuracy was improved by adjusting the hyperparameters.

[0055] S5. Obtain the image to be processed; input the image to be processed into the handwriting segmentation model, and the handwriting segmentation model outputs the prediction result. The prediction result (size 14*14*5) includes several bounding boxes (size 14*14*4) and the target confidence (size 14*14*1) indicating the probability that the target within the bounding box is text.

[0056] Example 2

[0057] Further, several bounding boxes output by the handwriting segmentation model are merged. The specific steps are as follows:

[0058] A1. Merge the bounding boxes that overlap in the Y-axis direction: As Figure 3 shown, if two bounding boxes do not overlap in the Y-axis direction but overlap in the X-axis direction (specifically, the Y-axis coordinate ranges of one bounding box coincide with those of another bounding box, and the X-axis coordinate ranges do not coincide), then the two bounding boxes are merged.

[0059] A2. Calculate the average value W and variance value S of the bounding box widths;

[0060] A3. Traverse the bounding boxes to find adjacent bounding boxes with a distance less than the distance threshold (the distance threshold in this embodiment is set to 0.5*W) in the X-axis direction; if multiple adjacent bounding boxes with a distance less than the distance threshold are found, preferentially merge the adjacent bounding boxes with the smallest distance; calculate the variance value S' of the merged bounding box width of these two adjacent bounding boxes. If S'<S, then merge these two adjacent bounding boxes, and recalculate and update the average value W and variance value S; otherwise, do not merge these two adjacent bounding boxes;

[0061] A4. Repeat step A3 until there are no more bounding boxes that can be merged. The merging result is as Figure 4 shown.

[0062] Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Claims

1. A handwriting segmentation method for electronic signatures based on recognition, characterized in that, Includes the following steps: A handwriting segmentation model is pre-built and trained, the handwriting segmentation model including a backbone network, the backbone network including several convolutional layers and several residual structures; Obtain the handwriting coordinate information of the user's electronic signature; generate a handwriting image based on the handwriting coordinate information; input the handwriting image as the image to be processed into the handwriting segmentation model; the handwriting segmentation model outputs the prediction result, which includes several bounding boxes and several target confidence scores. Merging the aforementioned bounding boxes includes: merging bounding boxes that overlap in the Y-axis direction, calculating the average and variance values ​​of the bounding boxes; traversing the bounding boxes, merging bounding boxes in the X-axis direction whose spacing is less than the spacing threshold and whose variance values ​​of the width of each bounding box decrease after merging, recalculating and updating the variance and average values ​​of the width of each bounding box, and continuing to attempt to merge bounding boxes until there are no more bounding boxes that can be merged. The handwritten handwriting segmentation results are obtained.

2. The handwriting segmentation method for electronic signatures based on recognition according to claim 1, characterized in that, The handwriting segmentation model also includes an input layer, a first convolutional layer, a second convolutional layer, and an output layer.

3. The handwriting segmentation method for electronic signatures based on recognition according to claim 1, characterized in that, The spacing threshold is determined based on the average width of each bounding box.

4. The handwriting segmentation method for electronic signatures based on recognition according to claim 1, characterized in that, The handwriting segmentation model is trained iteratively using a loss function, where the loss value calculated by the loss function is the sum of the target prediction loss value and the bounding box prediction loss value.

5. A handwriting segmentation device for electronic signatures based on recognition, characterized in that, Includes a processor and a memory for storing processor-executable instructions and a handwriting segmentation model; the processor is configured to execute the following instructions: Obtain the image to be processed; The input image to be processed is fed into the handwriting segmentation model, which includes a backbone network, which includes several convolutional layers and several residual structures; the handwriting segmentation model outputs prediction results, which include several bounding boxes and several target confidence scores. Merging the aforementioned bounding boxes includes: merging bounding boxes that overlap in the Y-axis direction, calculating the average and variance values ​​of the bounding boxes; traversing the bounding boxes, merging bounding boxes in the X-axis direction whose spacing is less than the spacing threshold and whose variance values ​​of the width of each bounding box decrease after merging, recalculating and updating the variance and average values ​​of the width of each bounding box, and continuing to attempt to merge bounding boxes until there are no more bounding boxes that can be merged. The handwritten handwriting segmentation results are obtained.

Images