Handwritten signature detection method and device and electronic product

Abstract

The invention provides a handwritten signature detection method and device and an electronic product, and the method comprises the steps: carrying out the joint prediction: predicting an input handwritten signature image through a detection head, outputting the parameters of a predicted rotating rectangular frame and a predicted horizontal rectangular frame, and carrying out the consistency constraint; generating a corresponding Gaussian attention map based on the predicted rotating rectangular frame parameter; carrying out point-by-point multiplication on the Gaussian attention map and an originally input handwritten signature image, and extracting a region of interest; the region of interest is sent to a classification head, a three-classification task is completed, three classification probability values are output, a rotating rectangular frame with the classification probability of the handwritten signature larger than a set threshold value is a detected handwritten signature frame, finally, redundant rotating rectangular frames are removed through maximum suppression, and a detection result is obtained. According to the method and the device, the problem of handwritten signature detection under target inclination, direction variation and non-signature handwritten signature interference in the prior art can be effectively solved.

Description

Technical Field

[0001] This application relates to the fields of image processing and pattern recognition technology, and more specifically to handwritten signature detection technology. Background Technology

[0002] In the current context of digital transformation, handwritten signature detection technology has become a key component of automated processes across many industries, particularly in the financial, legal, and government sectors. The core of this technology lies in accurately identifying and locating handwritten signatures in documents, providing a foundation for subsequent signature verification and comparison tasks. However, existing technologies face a series of challenges when dealing with signature detection tasks in natural scenarios, including but not limited to:

[0003] 1. The challenge of recognizing non-standard rectangular boxes

[0004] Traditional object detection algorithms, such as Faster R-CNN and the YOLO series, are primarily designed to detect targets with standard horizontal or vertical rectangular boundaries. However, in real-world applications, especially when signatures are photographed at an angle or when the writer's natural writing angle creates non-standard rectangular boxes, these algorithms struggle to accurately capture the actual boundaries of the signature. This results in overly broad detection boxes that include a large amount of irrelevant background information, thus affecting the accuracy of subsequent handwriting analysis.

[0005] 2. False detection issues caused by handwritten text interference

[0006] Besides the signature area, many documents also include other handwritten elements, such as dates, annotations, gender, and ethnicity information. These elements are extremely similar in form to handwritten signatures, significantly increasing the recognition difficulty for detection models. Existing detection methods often lack effective mechanisms to distinguish these similar but different handwritten contents, easily mistaking non-signature handwritten elements for signature areas, thus reducing the overall detection accuracy of the system.

[0007] 3. Lack of sensitivity to rotation angles

[0008] In signature detection tasks in natural scenes, the rotation angle of the signature is an extremely important factor. However, existing detection frameworks mostly focus on the horizontal or vertical layout of objects and lack sensitivity to changes in rotation angle. This not only limits the flexibility of detection but also increases the uncertainty in locating the signature region.

[0009] 4. The separation between classification and detection

[0010] Most existing technologies treat classification and detection as independent tasks. Even after detecting a candidate region, they cannot optimize the position and size of the detection box based on classification feedback. This separation leads to misclassification even if the detection box covers the signature region due to feature confusion within the box, and vice versa. This creates a disconnect between detection and classification, reducing the overall performance of the system.

[0011] 5. Limitations in dealing with complex backgrounds

[0012] In complex contexts, traditional feature statistics methods struggle to effectively distinguish signature regions from other text or patterns. Furthermore, these methods lack generalization ability when faced with diverse document formats and styles, are susceptible to background noise, and can lead to false positives and false negatives.

[0013] In summary, existing technologies perform poorly in handling handwritten signature detection in natural scenes, particularly when dealing with non-standard bounding boxes, handwritten interference, and sensitivity to rotation angles. Therefore, developing a new technological solution that can overcome these challenges and achieve high-precision signature detection is of paramount importance. Summary of the Invention

[0014] To address the key challenges of handwritten signature detection in natural scenes, such as target tilt, variable orientation, and interference from non-signature handwritten characters, this application proposes a handwritten signature detection method, device, and electronic product. It adopts a collaborative optimization framework that integrates dual-frame supervision and attention guidance to improve the accuracy and robustness of signature detection, especially in the case of complex backgrounds and multi-angle signatures.

[0015] The technical solution of this application is as follows:

[0016] In a first aspect of this application, a handwritten signature detection method is proposed, comprising:

[0017] Step 1, Joint prediction of the rotated rectangle and the horizontal rectangle: Input the handwritten signature image into the detection model, and the backbone network of the model outputs the parameters of the predicted rotated rectangle and the horizontal rectangle simultaneously through the detection box output head;

[0018] Step 2, Gaussian attention map generation: Generate the corresponding Gaussian attention map based on the predicted rotated rectangle parameters;

[0019] Step 3, Obtain the region of interest: Multiply the Gaussian attention map point by point with the original input handwritten signature image to extract the region of interest of the rotated rectangle;

[0020] Step 4, classify to obtain detection results: send the region of interest into the classification head to complete the classification task and output three classification probability values, corresponding to handwritten signature, handwritten non-signature and background respectively. Among them, the rotating rectangle with a classification probability of handwritten signature greater than the set threshold is the detected handwritten signature box. Finally, redundant rotating rectangles are removed by maximum suppression to obtain the detection result.

[0021] In some embodiments of this application, step 1 further includes preprocessing the handwritten signature image by introducing random angle rotation. Specifically, for the handwritten signature image, a portion of the signature on the signature image is randomly selected and rotated at a random angle. Various signature data with rotation angles are obtained through enhancement methods to enhance the model's adaptability and robustness to the tilted state of the handwritten signature.

[0022] In some embodiments of this application, the prediction parameters of the rotating rectangle include the center point coordinates, width, height, and rotation angle, and the prediction parameters of the horizontal rectangle include the width, height, and the center point coordinates shared with the rotating rectangle.

[0023] In some embodiments of this application, the Gaussian attention map generation in step 2 is to generate a Gaussian attention map using the parameters corresponding to each rotating rectangle output by the detection head. During generation, a normalized coordinate system is used to calculate the attention value, and the aspect ratio and rotation angle of the rotating rectangle are taken into account to form an asymmetric attention region consistent with the target direction.

[0024] In some embodiments of this application, the method for obtaining the region of interest in step 3 is as follows: The Gaussian attention map is multiplied point-by-point with the original input image, and the feature responses at different spatial locations in the original feature map are selectively enhanced or suppressed, thereby achieving spatial recalibration of the features and obtaining the corresponding region of interest for each predicted rotated bounding box; the calculation process can be described as follows: Let the input feature map be... The generated Gaussian attention map is The calibration feature map after dot product operation , where ⊙ represents position-by-position scalar multiplication; the weight value of each position in the Gaussian graph G is in the interval [0,1], which is the attn_value calculated above, where the position closer to the center of the rotating rectangle has a higher weight, forming an attention distribution that smoothly transitions from the center to the edge.

[0025] In some embodiments of this application, step 4 specifically includes: first, obtaining the two-dimensional vector of the region of interest.

[0026] Convert to a one-dimensional vector The data is then fed into a classification layer consisting of two fully connected layers. Finally, the softmax function outputs the probability values ​​for the three categories, corresponding to the class probabilities of handwritten signature, handwritten non-signature, and background, respectively.

[0027] In some embodiments of this application, the prediction model is trained based on steps 1-4. During training, the gradients of the predicted rotated and horizontal bounding boxes are calculated using the cross-union loss, and the prediction results of the two boxes are optimized simultaneously by using the consistency loss gradient. At the same time, closed-loop feedback optimization is performed after the three-class classification task: the multi-class cross-entropy loss of the three-class classification task is calculated, the classification result is optimized, the gradient of the classification result is backpropagated to the detection parameters of the rotated and horizontal bounding boxes, and the parameters are adjusted to achieve closed-loop feedback optimization of detection and classification interaction until the detection model converges and a trained detection model is obtained.

[0028] In some embodiments of this application, during backpropagation in training, the loss gradient of the classification task flows through the attention map, directly guiding the parameter adjustment of the rotating rectangle to achieve co-optimization between the classification and detection tasks.

[0029] This application also provides a handwritten signature detection device, which includes:

[0030] The prediction box unit is used to input the handwritten signature image into the detection model and simultaneously output the parameters of the predicted rotated rectangle and horizontal rectangle through the detection box output head.

[0031] A Gaussian attention map generation unit generates a Gaussian attention map based on the parameters of the rotated rectangle;

[0032] The region of interest acquisition unit performs point-by-point multiplication between the Gaussian attention map and the original input image to accurately extract the region of interest of the rotated rectangle;

[0033] The classification unit performs a three-class classification task, sending the rotated rectangle and the region of interest within the horizontal rectangle into the classification head to complete the final classification task and output three classification probability values, corresponding to handwritten signature, handwritten non-signature, and background, respectively. Among them, the rotated rectangle with a classification probability greater than the threshold for handwritten signature is the detected handwritten signature box. Finally, redundant rotated rectangles are removed by maximum suppression to obtain the detection result.

[0034] The prediction box unit also enhances the preprocessing of handwritten signature images by introducing random angle rotation. Specifically, for the handwritten signature image, a portion of the signature is randomly selected and rotated at a random angle. Various signature data with rotation angles are obtained through this enhancement method to improve the model's adaptability and robustness to the tilted state of the handwritten signature. The prediction parameters of the rotating rectangle include the center point coordinates, width, height, and rotation angle. The prediction parameters of the horizontal rectangle include the width, height, and the center point coordinates shared with the rotating rectangle.

[0035] The Gaussian attention map generation unit generates a Gaussian attention map using the parameters corresponding to each rotating rectangle output by the detection box output head. During generation, a normalized coordinate system is used to calculate the attention value, and the aspect ratio and rotation angle of the rotating rectangle are taken into account to form an asymmetric attention region consistent with the target direction.

[0036] The region of interest acquisition unit performs point-by-point multiplication between the Gaussian attention map and the original input image, selectively enhancing or suppressing the feature responses at different spatial locations in the original feature map, thereby achieving spatial recalibration of the features and obtaining the region of interest corresponding to the predicted rotated bounding box.

[0037] The classification task unit first obtains the two-dimensional vector of the region of interest. Convert to one-dimensional direction The data is then fed into a classification layer consisting of two fully connected layers. Finally, the softmax function outputs the probability values ​​for the three categories, corresponding to the probabilities of handwritten signature, handwritten non-signature, and background, respectively.

[0038] This application further provides an electronic product, including a processor and a memory, wherein the memory stores instructions that, when executed by the processor, constitute the above-mentioned handwritten signature detection method steps.

[0039] The advantages of this application include at least the following:

[0040] 1. This application employs a classification and detection separation method. By introducing a Gaussian attention mechanism and generating a Gaussian attention map through rotating a bounding box, it can focus on key regions of the signature, achieving higher classification accuracy. Then, the detection results are further optimized through feedback.

[0041] 2. This application supports signature box detection with rotation angle, and further optimizes the prediction results of the rotating rectangle by simultaneously supervising the rotating rectangle and the horizontal rectangle.

[0042] As can be seen, the detection method proposed in this application significantly improves the accuracy and speed of handwritten signature recognition, especially when processing tilted or distorted signature images. This technology has broad application prospects in scenarios such as financial and legal document verification, and can effectively prevent the risk of forged signatures and ensure information security. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0044] Figure 1 Signature detection network overall flowchart;

[0045] Figure 2 A schematic diagram of a rotating rectangle and a horizontal rectangle;

[0046] Figure 3 Gaussian attention maps from various angles;

[0047] Figure 4 The handwritten signature detection result. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] Explanation of some technical terms in this application:

[0051] OpenCV: An open-source image processing library;

[0052] YOLO: A one-stage detection network, which has now evolved to YOLO13;

[0053] FasterRCNN: A two-stage detection network;

[0054] IoU: Intersection over Union (IoU) ratio.

[0055] This application proposes a handwritten signature detection method that employs a collaborative optimization framework integrating dual-frame supervision and attention guidance. The following section first systematically elaborates on the two core innovative mechanisms within this framework: an angle-aware model based on joint prediction of rotated and horizontal rectangular frames, and a classification system implemented through a Gaussian attention mechanism.

[0056] 1. Angle prediction mechanism based on dual-frame joint supervision (angle-aware model)

[0057] In natural scenes, signatures are often tilted at multiple angles due to shooting conditions and writing habits, making it difficult for traditional horizontal rectangular boxes to fit closely, thus affecting subsequent handwriting analysis. To address this issue, this application reconstructs the detection box output head to simultaneously output the parameters of both rotated and horizontal rectangular boxes. Furthermore, a consistency constraint loss is added to both types of boxes during model training to mitigate the bias that may arise from predicting only the rotated rectangular box.

[0058] 2. A classification system incorporating Gaussian attention guidance

[0059] Inaccurate classification is the root cause of false detections of handwritten non-signatures such as dates. This application innovatively utilizes the output of rotated bounding boxes to generate Gaussian attention maps, constructing a closed-loop system for classification-guided detection optimization. By effectively suppressing background noise interference outside the rotated bounding boxes, the classifier focuses on the core semantic information within the boxes, thus clearly distinguishing between "handwritten signatures," "handwritten non-signatures" (such as dates, annotations, gender, etc.), and "background." During model training, the gradient of the higher-precision classification loss flows through the attention map during backpropagation and directly affects the parameters of the five rotated bounding boxes that generated it, indicating how the rotated bounding boxes should be fine-tuned to better serve the classification task. This system achieves a win-win situation for detection and classification: the classification task gains significantly higher accuracy due to the pure, high signal-to-noise ratio features provided by detection; the detection task receives semantic-level feedback from classification, enabling continuous iteration and optimization of localization results. The two form a virtuous cycle of synergistic enhancement, achieving a significant dual benefit.

[0060] In one embodiment of this application, the handwritten signature detection method generally includes the following four steps:

[0061] Step 1, Joint prediction of the rotated rectangle and the horizontal rectangle: Input the handwritten signature image into the detection model, and output the parameters of the predicted rotated rectangle and the horizontal rectangle simultaneously through the detection output head.

[0062] Step 2, Gaussian attention map generation: Generate the corresponding Gaussian attention map based on the predicted rotation rectangle parameters; form an asymmetric attention region consistent with the target direction, thereby achieving a high degree of focus on the core information within the box.

[0063] Step 3, Obtain the region of interest: Multiply the Gaussian attention map point by point with the original input handwritten signature image to accurately extract the region of interest of the rotated rectangle.

[0064] Step 4, Three-classification task: The region of interest is fed into the classification head to complete the final classification task, outputting three classification probability values, corresponding to handwritten signature, handwritten non-signature, and background, respectively. Among them, the rotated rectangles whose classification probability of handwritten signature is greater than a set threshold are the detected handwritten signature boxes. Finally, redundant rotated rectangles are removed by maximum suppression to obtain the detection result.

[0065] The prediction model used in this invention is trained based on steps 1-4. During training, the gradients of the predicted rotated and horizontal bounding boxes are calculated using the cross-union loss, and the prediction results of the two boxes are optimized simultaneously by using the consistency loss gradient. In addition, closed-loop feedback optimization is performed after the three-class classification task: the multi-class cross-entropy loss of the three-class classification task is calculated to optimize the classification results. The gradient of the classification results is backpropagated to the detection parameters of the rotated bounding box, and the parameters are adjusted to achieve closed-loop feedback optimization of detection and classification interaction until the detection model converges, resulting in a well-trained detection model.

[0066] The technical solution of this embodiment solves the key problems of target tilt, variable orientation, and interference from non-signature handwriting in handwritten signature detection. By independently predicting the parameters of the rotating and horizontal bounding boxes and applying consistency constraints, the detection accuracy of signatures at different angles is effectively improved. The generated Gaussian attention map focuses on the area within the rotating bounding box, and by multiplying the original image point by point, the features of the signature region are accurately extracted, reducing background interference. This region is then fed into the classification head to complete a three-class classification task, improving the ability to distinguish between signatures, non-signature handwriting, and the background. Finally, redundant detection boxes are removed using maximum suppression technology, ensuring accurate recognition of handwritten signatures and effectively solving the detection problems in existing technologies.

[0067] In addition, the closed-loop feedback mechanism used during model training, which calculates the multi-class cross-entropy loss to guide the parameter optimization of the rotating rectangle, realizes the interactive reinforcement of detection and classification, and significantly enhances the robustness and accuracy of the model.

[0068] The following further examples illustrate the specific implementation of the entire method:

[0069] I. Overall Framework:

[0070] The overall framework of the handwritten signature detection method proposed in this application is as follows: Figure 1 As shown.

[0071] First, the input signature image is preprocessed, and its size is uniformly adjusted to a fixed specification (W × H) while maintaining the original aspect ratio, where the number of channels C corresponds to the BGR three-channel format. This scheme defaults to setting both W and H to 640 pixels. The preprocessed image will then serve as the input to the backbone network. The backbone network offers good flexibility, and mature architectures such as the YOLO series, VIT, or Res series can be used for feature extraction.

[0072] The detection output head at the network's end is designed to simultaneously predict two types of bounding boxes: a rotated rectangle and a horizontal rectangle. The prediction parameters for the rotated rectangle include its center point coordinates, width, height, and rotation angle. The horizontal rectangle shares its center point with the rotated rectangle, so only its width and height need to be predicted. A consistency constraint loss is then added to both types of boxes to further improve the prediction accuracy of the rotated rectangle. This dual-box joint supervision mechanism effectively enhances the accuracy of signature localization for irregular shapes and mitigates the bias problems that may arise from predicting only the rotated rectangle.

[0073] Subsequently, based on the predicted rotating rectangle, the algorithm generates a corresponding Gaussian attention map. This generation process supports the incorporation of rotation angle information, thereby forming an asymmetric attention region consistent with the target direction.

[0074] Subsequently, by multiplying this attention map point by point with the original input image, the region of interest within the rotated rectangle is accurately extracted. This operation can effectively filter out background noise and retain the effective feature data within the region.

[0075] Finally, the extracted features are fed into a dedicated classification head to complete the final classification task. This classifier outputs a three-class result, covering the categories of "handwritten signature," "handwritten unsigned," and "background." By introducing this explicit sub-categorization, the model is forced to learn discriminative features between handwritten signatures and handwritten unsigned, thereby significantly reducing the false detection rate of common distractors such as dates.

[0076] II. Model Training 1: Definition of Rotated and Horizontal Rectangles

[0077] In deep learning-based signature detection tasks, directly using horizontal rectangles often fails to accurately fit the naturally tilted signature area due to factors such as shooting angle. Using rotated rectangles for annotation not only more accurately defines the target and reduces background interference, but also provides a more reliable region of interest for subsequent higher-level tasks such as handwriting comparison. This application uses the rotated rectangle representation defined in the OpenCV library, with a parameter system of (cx, cy, width, height, angle), where the rotated rectangle and horizontal rectangle are represented as follows: Figure 2 As shown. Specifically:

[0078] (cx, cy) represents the x and y coordinates of the center point of the rotated rectangle.

[0079] width: The length of the side that first coincides with the rectangle in the positive x-axis direction during clockwise rotation.

[0080] height: The length of the side adjacent to width.

[0081] angle: represents the angle taken from the positive x-axis direction, rotating clockwise until it coincides with the width side. This scheme uses radians, and its value range is limited to (0, π / 2).

[0082] At the same time, the model also predicts in parallel a horizontal rectangle that shares the center point (cx, cy) with the rotated rectangle. This rectangle can be fully described by only two parameters: width bbox_w and height bbox_h. The actual annotation information can be derived from the parameters of the rotated rectangle to obtain the width and height (bbox_w, bbox_h) of the corresponding horizontal rectangle through a predetermined geometric conversion formula, which is shown below.

[0083]

[0084] .

[0085] 2. Data labeling

[0086] During the data annotation phase, the LabelMe tool was used to perform fine-grained annotations on the collected signature images. For each handwritten signature instance, the annotators used a four-point method to define its area, with the four vertices labeled clockwise as top left, top right, bottom right, and bottom left. After obtaining the coordinates of these four points, the corresponding five rotated rectangle parameters (cx, cy, width, height, angle) could be automatically calculated using OpenCV's minAreaRect function. Then, using the aforementioned conversion formula, the bbox_w and bbox_h annotation information could be obtained.

[0087] 3. Randomly enhance the rotating rectangle.

[0088] For the training signature images, a portion of the signatures on the images is randomly selected and rotated at random angles. This augmentation method yields a variety of signature data with rotation angles, increasing the richness of the data and enabling the model to handle signature data from various angles. The specific implementation method is as follows:

[0089] First, the input annotation information is parsed to obtain all the signature rotation rectangles in the image. Then, based on a preset sampling probability, a portion of the signature rectangles in the image are randomly selected as augmentation targets.

[0090] For each selected signature box, the algorithm will expand the original area by 4 times (or even more) with that area as the center before cropping it. This is to preserve sufficient background context information, provide room for subsequent rotation transformations, and avoid improper truncation of the signature body's edges due to rotation.

[0091] Next, a random rotation angle θ is generated independently for each signature frame. Using the cv2.getRotationMatrix2D function in the OpenCV library, the rotation matrix is ​​calculated with the center of the signature frame as the origin, and a counterclockwise rotation operation is performed to obtain a new signature image block with the orientation adjusted.

[0092] After the rotation operation is completed, the horizontal bounding box of the new signature image block is calculated. To ensure the natural integration of the enhanced signature, the size of this bounding box is increased by a factor of r, where r is a random number between 1.5 and 3.0. This expansion operation effectively ensures that the edges of the rotated signature retain an appropriate amount of the original background, simulating the padding effect in a real scene and significantly reducing the risk of artifacts introduced by rotational filling (such as black borders or transparent areas).

[0093] Finally, the rotated and resized signature area is re-synthesized to its corresponding position in the original image, completing a local enhancement. This process efficiently introduces rotational variants while maintaining the authenticity of the signature shape.

[0094] In this embodiment, a preprocessing technique involving random angle rotation is introduced during the training of the signature detection model. This technique rotates selected portions of the signature image at random angles, creating more diverse signature samples that more closely resemble the rotation angle distribution in the real world. This significantly enhances the model's adaptability and robustness when faced with tilted signatures. This preprocessing increases the diversity of training data, enabling the model to learn more comprehensively from handwritten signatures at various angles. This ensures that the model can not only recognize signatures in standard poses but also accurately handle tilted signature instances caused by shooting angles or writing habits, greatly improving the model's generalization performance in practical applications. Furthermore, by adjusting the fusion method between the rotated signature frame and the original background, the model learns to establish a more precise spatial relationship between the rotated signature and its surrounding environment, further improving detection accuracy.

[0095] 4. Consistency constraints between rotated rectangle and horizontal rectangle

[0096] The detection head is designed to simultaneously output parameters for both the rotated bounding box and its horizontal bounding box. Specifically, the network predicts seven key parameters for each candidate region: the center coordinates (pred_cx, pred_cy), width (pred_width), height (pred_height), and rotation angle (pred_angle) of the rotated bounding box, and the width (pred_bbox_w) and height (pred_bbox_h) of the horizontal bounding box. Independent supervision signals are applied to the model to assess the localization accuracy of these two bounding boxes. Both the rotated and horizontal bounding boxes can be constrained using various loss functions widely used in modern object detection, such as IoU loss, GIoU loss, and CIoU (Intersection over Union) loss.

[0097] However, relying solely on independent supervision is insufficient to fully uncover the inherent geometric correlation between two rotated bounding boxes and a horizontal bounding box. To address this, we introduce a novel consistency constraint loss. The core idea of ​​this constraint is that the bounding rectangle of an ideally predicted rotated bounding box (i.e., the calculated horizontal bounding box) should spatially be highly consistent with the horizontal bounding box directly predicted by the network. The implementation path is as follows: First, based on the parameters of the predicted rotated bounding box (pred_cx, pred_cy, pred_width, pred_height, pred_angle), the theoretical width and height of its horizontal bounding box (cal_pred_bbox_w, cal_pred_bbox_h) are derived using the aforementioned geometric transformation formula. Then, this calculated theoretical width and height, along with the width and height of the horizontal bounding box directly predicted by the network (pred_bbox_w, pred_bbox_h), constitute an auxiliary supervision pair. The differences between them can also be measured by the aforementioned crossover ratio loss, thus forming a strong internal consistency constraint. This provides an additional regularization term derived from geometric consistency for the regression of the rotated rectangle, effectively avoiding logical fallacies in the spatial relationship between the rotated rectangle and the horizontal rectangle, and ensuring the accuracy and rationality of the prediction results of the rotated rectangle itself.

[0098] In this embodiment, a consistency constraint mechanism is innovatively applied to the joint prediction of the rotated bounding box and the horizontal bounding box. The introduction of this consistency constraint aims to ensure that the rotated bounding box and its circumscribed horizontal bounding box maintain a high degree of spatial consistency, thereby enhancing the predictive ability of the rotated bounding box, avoiding logical fallacies, and improving the accuracy and rationality of object detection.

[0099] 5. Optimization of false detection due to interference from non-signature handwriting

[0100] In signature detection tasks, false positives caused by non-signature handwriting interference such as dates and annotations are fundamentally due to the model's failure to accurately distinguish between signatures and non-signature handwriting at the feature level. To address this challenge, during the annotation phase, we not only annotate the signature area but also other interfering handwriting, constructing a clear three-class classification task (handwritten signature, handwritten non-signature, background). This forces the model to learn more discriminative features, rather than simply performing a binary classification of foreground and background.

[0101] The core of our proposed method lies in using a closed-loop feedback mechanism—where detection guides classification, and then classification optimizes detection—to simultaneously improve the accuracy of localization and classification. The implementation process of this method is as follows:

[0102] (1) Gaussian attention map generation

[0103] After passing through the backbone network, the detector head outputs multiple predicted rotated bounding boxes, each corresponding to a set of normalized parameters (pred_cx, pred_cy, pred_width, pred_height, pred_angle). Subsequently, a Gaussian attention map is generated using these five parameters. The generation method of this map can be defined by the following formula:

[0104] Establish an image coordinate system with the top-left corner as the origin, the positive x-axis pointing to the right, and the positive y-axis pointing downwards. Obtain the coordinates of each pixel at the model input size. Let i and j represent the coordinates on the x-axis and y-axis, respectively. Then, normalize according to the input dimensions W and H to obtain... Then, subtract the predicted 'cx' from the x-axis and the predicted 'cy' from the y-axis to obtain new coordinates based on the predicted center point. ,

[0105]

[0106]

[0107] Obtain the rotated coordinates using the rotation matrix As shown below

[0108]

[0109]

[0110] Generate the Gaussian attention map corresponding to the predicted rotated bounding box, and the attention value attn_value for each point is shown below.

[0111]

[0112] .

[0113] like Figure 3 The image shows some visualizations of Gaussian attention maps. From left to right, these are the visualizations of center point pred_cx=10, pred_cy=10, width and height pred_width=6, pred_height=10, and rotation angles pred_angle of 0, 10, 30, 45, 60, and 90 degrees, respectively.

[0114] In this embodiment, the generation of the Gaussian attention map fully utilizes the parameters of each rotated rectangle output by the detection head. Through coordinate transformation and Gaussian weight calculation, an asymmetric attention map aligned with the target direction is formed. First, based on the predicted center point position and rotation angle of the rotated rectangle, the pixel coordinates of all input feature maps are normalized, followed by a rotation alignment operation to ensure that the attention mechanism accurately reflects the orientation information of the rotated rectangle. The calculation of the attention value comprehensively considers the size and shape of the rotated rectangle, ensuring that features within the box are enhanced and interference outside the box is suppressed. Especially when dealing with non-standard rectangular tilted signatures, this method can more accurately capture the core features of the signature and reduce the influence of background noise. Furthermore, by adjusting the standard deviation of the Gaussian distribution, the concentration of attention can be flexibly controlled to adapt to signature instances of different sizes and complexities. This mechanism not only improves the accuracy of classification but also further optimizes the regression of the rotated rectangle through gradient updates via backpropagation, thereby establishing an effective closed-loop feedback system between the detection and classification tasks and significantly improving the overall performance of handwritten signature detection.

[0115] (2) Obtain the region of interest

[0116] For each predicted rotated bounding box, a corresponding region of interest is generated. By multiplying a Gaussian attention map with the model's input feature map, the feature responses at different spatial locations in the original feature map are selectively enhanced or suppressed, thereby achieving spatial recalibration of the features. Specifically, the calculation process can be described as follows: Let the input feature map be... The generated Gaussian attention map is The calibration feature map after dot product operation , where ⊙ represents position-wise scalar multiplication. The weight value of each position in the Gaussian graph G is in the interval [0,1], which is the attn_value calculated above. The closer the position is to the center of the rotating rectangle, the higher the weight, forming an attention distribution that smoothly transitions from the center to the edge.

[0117] In this embodiment, by performing point-by-point multiplication between the generated Gaussian attention map and the input feature map, selective enhancement or suppression of feature responses at different spatial locations in the input feature map can be achieved, thereby realizing spatial recalibration of features and effectively focusing on important information within the rotated bounding box. This technical solution not only strengthens the model's recognition of handwritten signature regions but also significantly improves the accuracy of classification. Furthermore, the closed-loop feedback mechanism between classification and detection optimizes the regression results of the rotated bounding box, greatly reducing false detections of non-signature handwritten characters and achieving synergistic optimization of detection and classification.

[0118] (3) Classification Head

[0119] First, the two-dimensional vector of the region of interest is obtained through the Flatten operation. Convert to a one-dimensional vector The data is then fed into the final classification layer. This layer consists of two fully connected layers. Two layers were chosen because the non-linear transformation of the classification layer is intended to enhance the model's expressive power. Finally, the softmax function outputs the probability values ​​for the three categories: "handwritten signature," "handwritten unsigned," and "background." Because the input features are pure, the classification head can more effectively compare and distinguish whether the content within the bounding box is a handwritten signature, for example, by using aspect ratio, content information, etc., to differentiate between handwritten signatures and other types of information.

[0120] By continuously optimizing classification accuracy through multi-class cross-entropy loss, this more precise loss gradient, during backpropagation, traces back along the computation graph, updating not only the weights of the classification layer itself but also flowing through the Gaussian attention map, affecting the original five rotated bounding box parameters (pred_cx, pred_cy, pred_width, pred_height, pred_angle). The gradient acts as a feedback signal, clearly indicating: "To make the classification more accurate, the rotated bounding box should be slightly adjusted to the left to include the complete signature," or "The width should be reduced to exclude the date digits on the right." In this way, the classification task is no longer merely a passive output of detection but transforms into an active "mentor" guiding the rotated bounding box to make fine adjustments. This design advantage is twofold: for classification, it gains "spatial prior" from the rotated bounding box, enabling it to work in low-noise environments and significantly improving accuracy; for detection, it gains "semantic feedback" from classification, allowing it to adjust its position and shape as if precisely navigated.

[0121] In this embodiment, during the backpropagation step, the loss gradient of the classification task is not only used to update the parameters of the classification head, but also flows backward through the Gaussian attention mechanism and directly affects the prediction parameters (pred_cx, pred_cy, pred_width, pred_height, pred_angle) of the rotated rectangle. This innovative design achieves deep interaction and collaborative optimization between classification and detection tasks. Specifically, by directly transmitting the gradient signal of the classification loss to the detection stage, the model can adjust the position and size of the rotated rectangle based on the requirements of classification performance to more accurately match the shape of the handwritten signature, especially when facing the detection challenge of non-standard rectangles. In this way, the guidance of classification prompts the rotated rectangle to be continuously fine-tuned until the features within the box are sufficient to support the correct classification decision, and vice versa. More accurate rotational rectangle generation helps the classifier make more accurate judgments. This two-way feedback mechanism significantly improves the robustness and adaptability of the model, ensuring that even in complex backgrounds and situations where multiple handwriting styles coexist, the handwritten signature region can be stably and accurately identified, effectively reducing the false detection rate and improving detection efficiency.

[0122] III. Model Prediction

[0123] Through the preceding iterative optimizations, the model eventually converges to a stable state, at which point the loss remains essentially unchanged. The trained weights are then fixed and used for model prediction. During prediction, the rotated bounding box outputs the probability of the corresponding handwritten signature class. Rotated bounding boxes with a probability greater than a certain threshold are retained. Finally, redundant signature boxes are removed using maximum suppression.

[0124] Figure 4 These are some visualizations of the test results.

[0125] In a further embodiment of this application, a handwritten signature detection device is also provided to implement the above-mentioned detection method of this application. The device includes the following functional units:

[0126] The prediction box unit takes the handwritten signature image input to the detection model and simultaneously outputs the parameters of the predicted rotated and horizontal rectangles through the detection box output head. The predicted parameters of the rotated rectangle include the center point coordinates, width, height, and rotation angle. The predicted parameters of the horizontal rectangle include the width, height, and the center point coordinates shared with the rotated rectangle. This prediction box unit also enhances the preprocessing of the handwritten signature image by introducing random angle rotation. Specifically, for the handwritten signature image, a portion of the signature is randomly selected and rotated at a random angle. This enhancement method obtains various signature data with rotation angles, thereby improving the model's adaptability and robustness to tilted handwritten signatures.

[0127] The Gaussian attention map generation unit generates a Gaussian attention map based on the parameters of the rotating rectangle. Specifically, it generates a Gaussian attention map using the parameters corresponding to each rotating rectangle output by the detection box output head. During generation, a normalized coordinate system is used to calculate the attention value, and the aspect ratio and rotation angle of the rotating rectangle are taken into account to form an asymmetric attention region consistent with the target direction.

[0128] The region of interest (ROI) acquisition unit is used to perform point-by-point multiplication of the Gaussian attention map with the original input image to accurately extract the ROI of the rotated bounding box. Specifically, the Gaussian attention map is multiplied with the original input image point-by-point, and the feature responses at different spatial locations in the original feature map are selectively enhanced or suppressed, thereby achieving spatial recalibration of the features and obtaining the ROI corresponding to the predicted rotated bounding box.

[0129] The classification unit performs a three-class classification task, feeding the rotated rectangle and the region of interest (ROI) within the horizontal rectangle into the classification head to complete the final classification task and output three classification probability values, corresponding to handwritten signature, handwritten non-signature, and background, respectively. Rotated rectangles with a classification probability greater than a threshold for handwritten signatures are considered detected handwritten signature frames. Finally, redundant rotated rectangles are removed using maximum suppression to obtain the detection result. The classification task unit first processes the acquired two-dimensional vector of the ROI. Convert to a one-dimensional vector The data is then fed into a classification layer consisting of two fully connected layers. Finally, the softmax function outputs the probability values ​​for the three categories, corresponding to the probabilities of handwritten signature, handwritten non-signature, and background, respectively.

[0130] This invention achieves accurate detection and classification of handwritten signatures in natural scenes through a collaborative optimization framework that integrates dual-frame supervision and attention guidance. An angle-aware model based on joint prediction of rotated and horizontal rectangles effectively addresses the issue of target tilt. By constructing a classification-detection closed-loop feedback system, Gaussian attention maps are used to optimize for interference from non-signature handwriting. This technical solution overcomes the limitations of traditional detection methods in complex scenarios, improves detection accuracy and efficiency, and is applicable to signature verification in fields such as finance and law, effectively avoiding security risks caused by detection errors.

[0131] Embodiments of this application also provide an electronic product, including a processor and a memory, wherein the memory stores instructions that, when executed by the processor, cause the product to perform the handwritten signature detection method described in the foregoing embodiments.

[0132] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0133] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting handwritten signatures, characterized in that, include: Step 1, Joint prediction of the rotated rectangle and the horizontal rectangle: Input the handwritten signature image into the detection model, and the backbone network of the model outputs the parameters of the predicted rotated rectangle and the horizontal rectangle simultaneously through the detection box output head; Step 2, Gaussian attention map generation: Generate the corresponding Gaussian attention map based on the predicted rotated rectangle parameters; Step 3, Obtain the region of interest: Multiply the Gaussian attention map point by point with the original input handwritten signature image to extract the region of interest of the rotated rectangle; Step 4, classify to obtain detection results: send the region of interest into the classification head to complete the classification task and output three classification probability values, corresponding to handwritten signature, handwritten non-signature and background respectively. Among them, the rotating rectangle with a classification probability of handwritten signature greater than the set threshold is the detected handwritten signature box. Finally, redundant rotating rectangles are removed by maximum suppression to obtain the detection result.

2. The handwritten signature detection method according to claim 1, characterized in that, Step 1 further includes preprocessing the handwritten signature image by introducing random angle rotation. Specifically, for the handwritten signature image, a portion of the signature on the signature image is randomly selected and rotated at a random angle. Various signature data with rotation angles are obtained through enhancement methods to improve the model's adaptability and robustness to the tilted state of the handwritten signature.

3. The handwritten signature detection method according to claim 1, characterized in that, The prediction parameters of the rotating rectangle include the center point coordinates, width, height, and rotation angle, while the prediction parameters of the horizontal rectangle include the width, height, and the center point coordinates shared with the rotating rectangle.

4. The handwritten signature detection method according to any one of claims 1 to 3, characterized in that, The Gaussian attention map generation in step 2 involves generating a Gaussian attention map using the parameters corresponding to each rotated rectangle output by the detection box output head. During generation, a normalized coordinate system is used to calculate the attention value, and the aspect ratio and rotation angle of the rotated rectangle are taken into account to form an asymmetric attention region consistent with the target direction.

5. The handwritten signature detection method according to any one of claims 1 to 3, characterized in that, The method for obtaining the region of interest in step 3 is as follows: multiply the Gaussian attention map with the original input image point by point, selectively enhance or suppress the feature responses at different spatial locations in the original feature map, thereby achieving spatial recalibration of the features and obtaining the region of interest corresponding to the predicted rotated rectangle.

6. The handwritten signature detection method according to claim 5, characterized in that, The calculation process for obtaining the region of interest in step 3 is as follows: Let the input feature map be... The generated Gaussian attention map is The calibration feature map after dot product operation , where ⊙ represents position-by-position scalar multiplication; the weight value of each position in the Gaussian graph G is in the interval [0,1], where the weight is higher the closer to the center of the rotating rectangle, forming an attention distribution that smoothly transitions from the center to the edge.

7. The handwritten signature detection method according to claims 1-4, characterized in that, Step 4 specifically includes: first, obtaining the two-dimensional vector of the region of interest. Convert to a one-dimensional vector The data is then fed into a classification layer consisting of two fully connected layers. Finally, the softmax function outputs the probability values ​​for the three categories, corresponding to the class probabilities of handwritten signature, handwritten non-signature, and background, respectively.

8. The handwritten signature detection method according to any one of claims 1 to 4, characterized in that the prediction model is trained based on steps 1 to 4, and during training, the gradients of the predicted rotated rectangle and horizontal rectangle are calculated using the cross-union loss, and the prediction results of the two boxes are optimized simultaneously by using the consistency loss gradient; at the same time, after the three-class classification task, closed-loop feedback optimization is also performed: the multi-class cross-entropy loss of the three-class classification task is calculated, the classification result is optimized, the gradient of the classification result is backpropagated to the detection parameters of the rotated rectangle, the parameters are adjusted, and closed-loop feedback optimization of the interaction between detection and classification is realized until the detection model converges, and the trained detection model is obtained.

9. The handwritten signature detection method according to claim 8 is characterized in that, during backpropagation in training, the loss gradient of the classification task flows through the attention map, directly guiding the parameter adjustment of the rotating rectangle, so as to achieve collaborative optimization between classification and detection tasks.

10. A handwritten signature detection device, characterized in that, include: The prediction box unit is used to input the handwritten signature image into the detection model and simultaneously output the parameters of the predicted rotated rectangle and horizontal rectangle through the detection box output head. A Gaussian attention map generation unit generates a Gaussian attention map based on the parameters of the rotated rectangle; The region of interest acquisition unit performs point-by-point multiplication between the Gaussian attention map and the original input image to accurately extract the region of interest of the rotated rectangle; The classification unit performs a three-class classification task, sending the rotated rectangle and the region of interest within the horizontal rectangle into the classification head to complete the final classification task and output three classification probability values, corresponding to handwritten signature, handwritten non-signature, and background, respectively. Among them, the rotated rectangle with a classification probability greater than the threshold for handwritten signature is the detected handwritten signature box. Finally, redundant rotated rectangles are removed by maximum suppression to obtain the detection result.

11. The handwritten signature detection device according to claim 10, characterized in that, The prediction box unit also enhances the preprocessing of handwritten signature images by introducing random angle rotation. Specifically, for the handwritten signature image, a portion of the signature is randomly selected and rotated at a random angle. Various signature data with rotation angles are obtained through this enhancement method to improve the model's adaptability and robustness to the tilted state of the handwritten signature. The prediction parameters of the rotating rectangle include the center point coordinates, width, height, and rotation angle. The prediction parameters of the horizontal rectangle include the width, height, and the center point coordinates shared with the rotating rectangle.

12. The handwritten signature detection device according to claim 10, characterized in that, The Gaussian attention map generation unit generates a Gaussian attention map using the parameters corresponding to each rotating rectangle output by the detection box output head. During generation, a normalized coordinate system is used to calculate the attention value, and the aspect ratio and rotation angle of the rotating rectangle are taken into account to form an asymmetric attention region consistent with the target direction.

13. The handwritten signature detection device according to claim 10, characterized in that, The region of interest acquisition unit performs point-by-point multiplication between the Gaussian attention map and the original input image, selectively enhancing or suppressing the feature responses at different spatial locations in the original feature map, thereby achieving spatial recalibration of the features and obtaining the region of interest corresponding to the predicted rotated bounding box.

14. The handwritten signature detection device according to claim 10, characterized in that, The classification task unit first obtains the two-dimensional vector of the region of interest. Convert to a one-dimensional vector The data is then fed into a classification layer consisting of two fully connected layers. Finally, the softmax function outputs the probability values ​​for the three categories, corresponding to the probabilities of handwritten signature, handwritten non-signature, and background, respectively.

15. An electronic product comprising a processor and a memory, wherein the memory stores instructions that, when executed by the processor, cause the device to perform the handwritten signature detection method of claims 1-9.

Images