A contract signature detection method and system based on improved YOLOv12

Abstract

The application belongs to the technical field of contract signature detection, and specifically discloses a contract signature detection method and system based on improved YOLOv12, which comprises the following steps: image preprocessing, improved YOLOv12 detection network and training optimization, input of the preprocessed enhanced image into the improved YOLOv12 detection network after training, and output of a prediction result and post-processing. On the basis of improving the YOLOv12 backbone network, the good balance between the detection speed and the accuracy is maintained through the design of the lightweight SAFE module and the SGDA module; a signature-oriented channel-space hybrid attention mechanism is proposed, and on the basis of the original three detection branches of YOLOv12, a high-resolution detection branch for extremely small signatures is newly added, and then the model is forced to focus on difficult samples through online difficult example mining, so that the whole method framework realizes the end-to-end optimization from image preprocessing, feature extraction to signature positioning, the detection speed and the accuracy are taken into account, and the robustness and the automation level of the contract signature detection in a complex scene are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of contract signature detection technology, specifically to a contract signature detection method and system based on an improved YOLOv12. Background Technology

[0002] In the contract document management process, signatures serve as crucial evidence of legal validity and authenticity. Their accurate detection and location are essential for automated auditing, contract verification, and risk control. However, contract images collected in actual business scenarios often suffer from complex background interference, tilted shooting angles, uneven lighting, inconsistent signature sizes, and a wide variety of counterfeit signature styles, posing significant challenges to signature detection.

[0003] Early signature detection methods relied heavily on traditional image processing techniques, such as color space segmentation, shape matching, or manually designed local features (e.g., SIFT, HOG). These methods were effective in controlled environments, but they were unable to handle complex situations in real-world scenarios, such as signatures being similar in color to the background, signature deformation, partial occlusion, and image quality degradation. Their generalization ability and robustness were clearly insufficient.

[0004] In recent years, the rapid development of deep learning technology, especially object detection algorithms, has provided new solutions for signature detection. General object detection frameworks such as the YOLO series, with their efficient end-to-end detection capabilities, have been attempted for signature detection tasks. However, existing methods typically treat signatures as ordinary targets during training, neglecting their unique visual attributes—such as translucency, fine texture, blurred edges, and frequent overlap with background text or patterns. This makes it difficult for detection models to effectively distinguish signatures from interference areas, leading to missed detections and false positives. Furthermore, the high cost of labeling signature samples, the imbalance between positive and negative samples, and the low detection accuracy of small-sized signatures also limit the practical deployment effectiveness.

[0005] Therefore, it is urgent to optimize the general detection model specifically for the characteristics of signature detection tasks, introduce prior knowledge of signature structure, enhance the ability to express fine-grained features, and combine high-quality annotation tools (such as LabelMe) to build an accurate signature dataset, so as to improve the accuracy and robustness of contract signature detection in complex scenarios. Summary of the Invention

[0006] To address the issues of low accuracy, frequent false negatives, and missed detections in contract document signature detection during automated auditing processes, caused by the complex visual characteristics of signatures (such as semi-transparency, fine textures, blurred edges, and overlap with background text or patterns) and variable shooting environments (such as uneven lighting, tilted angles, and resolution differences), this invention provides a contract signature detection method and system based on an improved YOLOv12. First, a high-quality signature dataset is constructed, using polygonal contour points to annotate signature areas, encompassing multi-source contract images and diverse signature styles. In the image preprocessing stage, a lightweight document region segmentation network is introduced to remove complex background interference, and CLAHE and gamma correction are used for adaptive image enhancement, effectively improving the contrast and recognizability of the signature area. A signature-aware fine-grained feature enhancement module (SAFE) is embedded in the YOLOv12 backbone network. This module adaptively samples the edge texture and irregular deformation of the signature area through parallel deformable convolutions v3 (with dilation rates of 1, 2, and 3), and expands the receptive field by combining multi-branch dilated convolutions to capture multi-scale contextual information of the signature, significantly improving the model's sensitivity to semi-transparent areas and blurred boundaries of the signature. Simultaneously, a signature-oriented dual attention fusion module (SGDA) is designed, cascading channel attention and spatial attention mechanisms. Channel attention filters feature channels related to the signature to suppress background text and texture interference, while spatial attention generates a signature foreground saliency heatmap to guide the model to focus on potential signature areas. Residual connections ensure lossless information transmission, strengthening the network's ability to distinguish foreground signatures from complex backgrounds. In the detection head, addressing the uneven distribution of signature sizes (coexistence of small-sized signatures and large-area official seals), a minimal signature detection branch (with input being a high-resolution feature map downsampled by 4 times) is added to the original three detection scales. An improved BiFPN structure is used for multi-scale feature weighted fusion. K-Means++ clustering generates 12 prior anchor boxes matching the signature shape, and a signature center point offset prediction branch is introduced to optimize the anchor point allocation strategy and loss function. The loss function employs multi-task joint optimization, including Focal Loss classification loss, CIoU localization loss, signature-aware edge loss, and center point offset loss. Online hard example mining forces the model to focus on difficult samples, and combined with combined data augmentation strategies and cosine annealing to restart the learning rate scheduling, the problem of imbalanced positive and negative samples is alleviated. In the post-processing stage, weighted non-maximum suppression is used to fuse overlapping detection boxes by confidence weight, improving detection continuity in scenarios with overlapping signatures and partial occlusion. The entire methodology achieves end-to-end optimization from image preprocessing and feature extraction to signature localization, balancing detection speed and accuracy, significantly improving the robustness and automation of contract signature detection in complex scenarios, and solving the problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a contract signature detection method based on improved YOLOv12, comprising the following steps: S1. Image preprocessing: S2. Improved YOLOv12 detection network and training optimization; S3. Input the preprocessed enhanced image into the trained improved YOLOv12 detection network and output the prediction result; S4. Post-process the output prediction results.

[0008] Preferably, step S1 specifically includes the following: S11. Take the contract document image as input, first perform size normalization processing, use anti-aliased bicubic interpolation algorithm to uniformly scale the image to 640×640 pixels, and convert it to RGB color space; S12. A lightweight document region segmentation network is introduced. The U-Net architecture with MobileNetV3-Small as the encoder is used to predict the binary mask of the document region. The network is trained by weighted sum of binary cross-entropy loss and soft IoU loss. After filling the mask holes by morphological closing operation, the original image and the mask are multiplied element by element to obtain the document foreground image. S13. Perform contrast-limited adaptive histogram equalization and gamma correction sequentially on the foreground image of the document to obtain the preprocessed enhanced image.

[0009] Preferably, in step S2, the improved YOLOv12 detection network specifically refers to the following: compared with the YOLOv12 backbone network, the improved YOLOv12 detection network further includes: a signature perception feature enhancement module, a signature-guided attention module, and a multi-scale detection head improvement and optimization module.

[0010] Preferably, the signature-aware feature enhancement module specifically includes: introducing a fine-grained signature-aware feature enhancement module SAFE into the YOLOv12 backbone network. Each SAFE module contains three parallel deformable convolutions v3 with different dilation rates. The deformable convolutions learn a two-dimensional offset and modulation scalar for each sampling point, enabling the receptive field to adaptively fit the irregular edges and curved textures of the signature. The output feature maps of the three branches are concatenated in the channel dimension, and then feature fusion and channel dimensionality reduction are performed through 1×1 convolution to output an enhanced feature map, which is then added to the original feature map through residual connections.

[0011] Preferably, the signature-guided attention module includes: embedding a signature-guided channel-spatial hybrid attention (SGDA) mechanism between the backbone network and the detection head. This mechanism cascades channel attention and spatial attention, and introduces residual connections to achieve deep fusion of original features and enhanced features; specifically including the following: S51, Channel Attention: First, the input feature map... F Simultaneously, global average pooling (GAP) and global max pooling (GMP) are performed to obtain two feature vectors, which are then fed into a shared two-layer fully connected network. The output of the two channel attention vectors is summed and activated by a sigmoid function to generate channel attention weights. ,Will and F Element-wise multiplication yields the channel recalibration features. ; S52, Spatial Attention: (This appears to be a fragment of a sentence and doesn't translate directly.) Average pooling and max pooling are performed along the channel dimension. The two generated 2D feature maps are then concatenated and spatial attention weights are generated by passing them through a 7×7 convolutional layer and sigmoid activation. ,Will and Element-wise multiplication yields spatially enhanced features. ; S53, Residual Linkage: [This part is incomplete and likely refers to a separate section or code.] F and Residual connections, output feature maps .

[0012] Preferably, the multi-scale detection head improvement and optimization module includes: adding a detection branch for extremely small signatures based on the original three detection scales; specifically including the following: S61. Improve and optimize the detection head of YOLOv12. On the basis of the original three detection scales, add a new detection branch for extremely small signatures, which is used to accurately detect extremely small signatures with an area of ​​less than 16×16 pixels. S62. An improved BiFPN structure is adopted, which introduces learnable weights in the bidirectional paths of top-down and bottom-up to perform weighted fusion of features at different scales and improve the expressive power of multi-scale features. S63. Redesign the size and proportion of the anchor boxes. Use the K-Means++ algorithm to cluster the statistical information of the signatures in the training dataset to generate 12 prior anchor boxes that match the shape of the signatures, and assign them to the 4 detection branches respectively. S64. The detection head outputs the category confidence of each anchor point box, the bounding box regression parameters, and the offset of the signature center point.

[0013] Preferably, the training optimization specifically includes: employing an end-to-end multi-task joint training strategy and designing a four-part loss function, including classification loss, localization loss, signature perception loss, and center point offset loss. During training, a combined data augmentation strategy is used in conjunction with the high-quality signature dataset annotated by LabelMe, and a cosine annealing restart strategy is employed to adjust the learning rate. Simultaneously, the online hard example mining OHEM is enabled to focus on difficult samples, and the top 30% of difficult examples in each batch are selected based on their loss values ​​for gradient backpropagation.

[0014] Preferably, the post-processing of the output prediction results specifically includes: applying weighted non-maximum suppression to the prediction results output by the detection head, setting the IoU threshold to 0.5; for a group of overlapping boxes, the fused new coordinates are the weighted sum of the coordinates of each box according to its confidence score, and the fused confidence score is the maximum confidence score within the cluster, effectively improving the detection continuity in cases of overlapping or partial occlusion of signatures; finally, the position coordinates, signature categories, signature center point coordinates, and confidence scores of all signatures in the contract image are output, and the results are saved in JSON format, while a visual detection image is generated.

[0015] On the other hand, to achieve the above objectives, the present invention also provides the following technical solution: a contract signature detection system based on an improved YOLOv12, comprising the following modules: Image preprocessing module: Taking the contract document image as input, it first performs size normalization, then uses an anti-aliased bicubic interpolation algorithm to uniformly scale the image to 640×640 pixels and converts it to RGB color space; a lightweight document region segmentation network is introduced, using a U-Net architecture with MobileNetV3-Small as the encoder to predict binary masks for document regions. It is trained by a weighted sum of binary cross-entropy loss and soft IoU loss. After morphological closing operation to fill the mask holes, the original image and the mask are multiplied element-wise to obtain the document foreground image; the document foreground image is then subjected to contrast-limited adaptive histogram equalization and gamma correction to obtain the preprocessed enhanced image; The signature-aware feature enhancement module introduces a fine-grained signature-aware feature enhancement module (SAFE) into the YOLOv12 backbone network. Each SAFE module contains three parallel deformable convolutions (v3) with different dilation rates. The deformable convolutions learn a two-dimensional offset and modulation scalar for each sampling point, enabling the receptive field to adaptively fit the irregular edges and curved textures of the signature. The output feature maps of the three branches are concatenated along the channel dimension, and then feature fusion and channel dimensionality reduction are performed through 1×1 convolutions to output an enhanced feature map. This enhanced feature map is then added to the original feature map through residual connections. Signature-guided attention module: A signature-guided channel-spatial hybrid attention (SGDA) mechanism is embedded between the backbone network and the detection head. This mechanism cascades channel attention and spatial attention and introduces residual connections to achieve deep fusion of original features and enhanced features. The multi-scale detection head improvement and optimization module adds a detection branch for extremely small signatures, based on the original three detection scales, to accurately detect signatures with an area smaller than 16×16 pixels; it adopts an improved BiFPN structure, introducing learnable weights in the bidirectional paths from top to bottom and bottom to top, and weightedly fuses features from different scales to improve the expressive power of multi-scale features; it redesigns the size and proportion of anchor boxes, and uses the K-Means++ algorithm to cluster the labeled statistical information of signatures in the training dataset, generating 12 prior anchor boxes that match the shape of the signatures, which are assigned to the four detection branches; the detection head outputs the class confidence, bounding box regression parameters, and offset of the signature center point for each anchor box; Training optimization module: An end-to-end multi-task joint training strategy is adopted, and a four-part loss function is designed, including classification loss, localization loss, signature perception loss, and center point offset loss. During training, a combined data augmentation strategy is used in conjunction with the high-quality signature dataset labeled by LabelMe, and a cosine annealing restart strategy is used to adjust the learning rate. At the same time, the online hard example mining OHEM is enabled to focus on difficult samples, and the top 30% of hard examples in each batch are selected according to their loss values ​​for gradient backpropagation. Post-processing module: The prediction results output by the detection head are subjected to weighted non-maximum suppression, and the IoU threshold is set to 0.5. For a group of overlapping boxes, the new coordinates after fusion are the weighted sum of the coordinates of each box according to its confidence score. The confidence score after fusion is the maximum confidence score within the cluster, which effectively improves the detection continuity in the case of overlapping or partially occluded signatures. Finally, the module outputs the position coordinates of all signatures in the contract image, the signature category, the coordinates of the signature center point, and the confidence score. The results are saved in JSON format, and a visual detection image is generated at the same time.

[0016] The beneficial effects of this invention are: 1) Signature perception multi-scale feature enhancement mechanism: A signature perception feature enhancement module is embedded after the 2nd, 3rd and 4th stages of the YOLOv12 backbone network. This module adaptively samples the edge texture and irregular deformation of the signature area through parallel deformable convolution, expands the receptive field by combining multi-branch dilated convolution, captures multi-scale contextual information of the signature, and performs feature fusion and dimensionality reduction through 1×1 convolution, effectively enhancing the model's response to semi-transparent areas and blurred boundaries of the signature.

[0017] 2) Signature-oriented dual attention fusion mechanism: A signature-oriented dual attention fusion module is embedded between the backbone network and the detection head. This module cascades channel attention and spatial attention. Channel attention uses global average pooling and global max pooling combined with a two-layer fully connected network to filter feature channels related to the signature. Spatial attention generates a signature foreground saliency heatmap through channel pooling and 7*7 convolution, and achieves deep fusion of original features and enhanced features through residual connections.

[0018] 3) Adaptive multi-scale detection head and anchor point optimization strategy: In view of the extremely uneven distribution of signature size, a detection branch for extremely small signatures is added on the basis of the original three detection scales of YOLOv12. An improved BiFPN structure is used for multi-scale feature weighted fusion. Twelve prior anchor point boxes matching the signature shape are generated by K-Means++ clustering. A signature center point offset prediction branch is introduced to provide structured information for subsequent signature verification and comparison.

[0019] 4) Multi-task joint training and hard example mining optimization framework: Construct a multi-task joint loss function including Focal Loss classification loss, CIoU localization loss, signature perception edge loss and center point offset loss. Adopt AdamW optimizer and cosine annealing to restart learning rate scheduling. Combined with combined data augmentation strategy and online hard example mining mechanism, it effectively alleviates the problem of imbalance between positive and negative samples and improves the generalization ability of the model.

[0020] 5) Weighted Non-Maximum Suppression Post-processing Strategy: Weighted Non-Maximum Suppression is used to replace the traditional NMS. For overlapping detection boxes with IoU exceeding the threshold (0.5), the coordinates are fused by weighting according to the confidence level. After fusion, the confidence level is taken as the maximum value within the cluster, which significantly improves the detection continuity in scenarios of overlapping signatures and partial occlusion. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the contract signature detection method based on the improved YOLOv12 in an embodiment of the present invention; Figure 2 This is a schematic diagram of the multi-scale feature enhancement backbone network for signature perception in an embodiment of the present invention; Figure 3 This is a schematic diagram of the experimental results of the model of the present invention; Figure 4 This is a schematic diagram of the detection results after interfering with the image; Figure 5 This is a schematic diagram of a contract signature detection system module based on the improved YOLOv12 in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention provides a technical solution: a contract signature detection method based on an improved YOLOv12, the method flow architecture of which is as follows: Figure 1 As shown, in this embodiment, all steps are implemented based on the Python programming language and the PyTorch deep learning framework, and the labeling tool used is LabelMe. The specific steps of the implementation process of this invention are as follows: Step 1: In constructing the dataset for intelligent image parsing and seal detection and recognition of contract documents, a large number of original contract document images were systematically acquired from enterprise CRM systems, historical contract archives, and on-site photos taken by frontline business personnel using mobile terminals. A total of 12,000 images were collected and semantically annotated at the pixel level, and divided into a training set of 9,600 images, a validation set of 1,200 images, and a test set of 1,200 images in an 8:1:1 ratio.

[0024] Step 2: Use the LabelMe tool to annotate the polygonal outline of the signature area in each contract image, and record the vertex coordinate sequence of each signature boundary. For semi-transparent seals, their fully visible outer contour is annotated. The annotation results are saved as a JSON file in COCO format, containing the image ID, seal category ID (official seal=1, financial seal=2, legal representative seal=3, seal across the seam=4), polygon vertex array, and bounding box. For signatures with an area smaller than 32×32 pixels, they are marked as a separate subclass called "Very Small Signature".

[0025] Step 3: Image size normalization and color space conversion: Scale the original contract image to a fixed size. Pixels. The scaling algorithm uses anti-aliased bicubic interpolation to preserve edge sharpness. Simultaneously, all images are converted from possible RGBA or grayscale modes to the RGB three-channel color space. For grayscale inputs, a single channel is copied to three channels.

[0026] Step 4: To eliminate the interference of complex non-document backgrounds (such as desktop textures, handheld devices, shadows, etc.) in the contract image on signature detection, a document region segmentation network was designed.

[0027] 4.1: A U-Net architecture with MobileNetV3-Small as the encoder is adopted. The encoder outputs feature maps with a stride of 32. The decoder gradually restores spatial resolution through skip connections.

[0028] 4.2 Loss Function: The training loss of the segmentation network is the binary cross-entropy loss. and loss Weighted sum: ; in This is a probability map of the predicted document regions. A mask for real documents. . Loss is defined as: ; 4.3 Post-mask processing: The predicted probability map is binarized with a threshold of 0.5 to obtain a binary mask. Morphological closing operations (kernel size) are employed. Fill in the tiny holes in the mask to ensure the document area remains continuous and intact. The final document foreground image is: ; Step 5: Adaptive image enhancement of the document foreground image Contrast and lighting equalization were performed to enhance the legibility of the signature area.

[0029] 5.1 CLAHE Enhancement: Adaptive histogram equalization with limited contrast is used to divide the image into 8*8 patches and limit the contrast magnification factor to 2.0 to prevent excessive noise amplification.

[0030] Gamma correction: Applying gamma transform to adjust the brightness curve: ; set up .

[0031] 5.2 Output: The enhanced image is denoted as... This serves as the input for the subsequent backbone network.

[0032] Step 6: Construct a signature-aware multi-scale feature enhancement backbone network. First, embed signature-aware feature enhancement modules after stages 2, 3, and 4 of the YOLOv12 backbone network, such as... Figure 2As shown, this invention introduces a fine-grained feature enhancement module for signature perception into the YOLOv12 backbone network, proposing the first improved YOLOv12 framework that integrates signature structure priors. By embedding deformable convolutions and multi-branch dilated convolutions at various stages of the backbone network, it not only preserves the edge texture details of the signature area but also expands the receptive field to capture multi-scale contextual information, effectively solving the problem of missed detection caused by the semi-transparency of the signature and background obfuscation.

[0033] 6.1 Improvements to Deformable Convolution: Each SAFE module contains 3 parallel deformable convolution layers with different dilation rates, and the kernel size... expansion rate Deformable convolution learns a two-dimensional offset for each sampling point. and a modulation scalar Its output is: ; in , These are the convolution weights; 6.2 Multi-branch feature fusion: The output feature maps of the three dilation rate branches are concatenated along the channel dimension to obtain... Subsequently, a 1x1 convolutional layer is used for feature fusion and channel dimensionality reduction to output an enhanced feature map. ; 6.3 Residual Linkage: [This section appears to be incomplete and requires further context.] Compared with the original input feature map Add them together to get the final output of the module.

[0034] Step 7: Signature-Oriented Dual Attention Fusion Module. A signature-oriented dual attention fusion module (SGDA) is embedded between the backbone network output and the detection head. This module cascades channel attention and spatial attention and introduces residual connections. This embodiment of the invention addresses the difficulty of distinguishing signatures from the background by proposing a signature-oriented channel-spatial hybrid attention mechanism. This mechanism first filters signature-related feature channels through channel attention (based on dual-path feature aggregation of GAP and GMP) to suppress interference from background text and textures; then, it utilizes spatial attention to generate a saliency heatmap of the signature foreground, guiding the model to focus on the potential signature area.

[0035] SGDA differs from ordinary CBAM: CBAM is a general module that treats all targets equally. This embodiment designs SGDA to generate a "signature foreground saliency heatmap" within spatial attention using cascaded channels and spatial attention, and leverages 7x7 large-kernel convolutions. Its explicit purpose is to "segment" potential signature regions from complex contract backgrounds (text, textures). This is an attention mechanism with structured priors, aiming at "localization," not just "weighting." This embodiment emphasizes that SGDA + residual connections is a signature region candidate generator, while CBAM is merely a feature recalibrator; their functional levels are also different.

[0036] 7.1 Channel Attention Submodule: Input Feature Map Simultaneously, global average pooling (GAP) and global max pooling (GMP) are performed to obtain two feature vectors. and They are fed into a shared two-layer fully connected network (hidden layer dimension). The output consists of two channel attention vectors, which are then summed and activated by a sigmoid function to generate the channel attention weights. ; Will and Element-wise multiplication yields the channel recalibration features. .

[0037] 7.2 Spatial Attention Submodule: For Average pooling and max pooling are performed along the channel dimension to generate two 2D feature maps. and After concatenating the two along the channel dimension, spatial attention weights are generated by passing them through a 7x7 convolutional layer and sigmoid activation. ; Will and Element-wise multiplication yields spatially enhanced features. .

[0038] 7.3 Residual Connections and Output: Final Output Feature Map Residual connections ensure the lossless transmission of original information, enhance the model's ability to focus on the foreground area of ​​the signature, and suppress interference from background text and textures.

[0039] Step 8: Construct an adaptive multi-scale detection head. To address the issue of extremely uneven distribution of signature sizes (from signatures as small as 8*8 pixels to official seals as large as 200*200 pixels), the detection head of YOLOv12 is improved.

[0040] 8.1 Add a branch for detecting extremely small signatures: In addition to the original three detection scales in YOLOv12, a fourth detection branch has been added. The input to this branch is the high-resolution feature map (downsampled by 4 times, size 160*160) output from the second stage of the backbone network, which is specifically used to detect extremely small signatures with an area of ​​less than 16*16 pixels.

[0041] 8.2 Feature Pyramid Fusion Enhancement: An improved BiFPN structure is adopted, introducing learnable weights to each detection branch in the bidirectional path from top to bottom and bottom to top, and performing weighted fusion of features at different scales. (The text then abruptly shifts to a different topic:) Taking layer features as an example: ; in For input features, This is the output of the top-down path. These are learnable weights.

[0042] Step 9: Optimization of Prior Anchor Boxes Based on K-Means Clustering To make the anchor points more closely match the size and shape distribution of real signatures, statistical clustering was performed on the width and height of all annotation boxes in the training set.

[0043] 9.1 Clustering Algorithm: The K-Means++ algorithm is used for initialization, and the number of clusters is... The distance metric function is defined as: ; in This is the intersection-union ratio of the two bounding boxes.

[0044] 9.2 Clustering Results: Clustering yielded 12 anchor box sizes, which were assigned to the 4 detection branches. The anchor box size assigned to the smallest signature branch was approximately... Small signature branch allocation Branch allocation upon signing the seal Large signature branch allocation

[0045] Step 10: Prediction of the offset of the signature center point In each detection head, besides predicting bounding box parameters In addition to category confidence, additionally predict the offset of the signature center point. This offset is defined as the normalized offset from the center of the predicted bounding box to the actual geometric center of the signature. It provides richer structured information for subsequent signature verification and comparison, laying a solid foundation for downstream tasks such as signature comparison and overlapping signature differentiation in automated contract auditing processes.

[0046] Step 11: Define the joint loss function for multiple tasks Total loss function It consists of four parts: classification loss, localization loss, signature perception edge loss, and center point offset loss. ; 11.1 Classification Loss Focal Loss is employed to mitigate the severe imbalance between positive and negative samples (signature vs. background). ; in , , This represents the probability of the correct class predicted by the model.

[0047] 11.2 Positioning Loss CIoU Loss is used, taking into account the overlap area of ​​the bounding boxes, the distance between the center points, and the consistency of the aspect ratio. ; in For Euclidean distance, These are the center points of the predicted bounding box and the ground truth bounding box, respectively. The length of the diagonal of the minimum bounding rectangle. For aspect ratio consistency parameters, This is a weighting factor.

[0048] 11.3 Signature Perception Edge Loss To predict the edges of the signature area, the Canny operator (low threshold 50, high threshold 150) is used to extract the ground truth edge map from the ground truth bounding box. The computational model predicts the feature map (after Sigmoid activation) and... L1 loss: ; Weighting coefficient .

[0049] 11.4 Center point offset loss Using Smooth L1 loss, the prediction of the signature center point offset is supervised. ; Weighting coefficient .

[0050] Step 12: Optimizer and Learning Rate Scheduling: 12.1 Optimizer: Using the AdamW optimizer, initial learning rate Weight decay 5*10 -4 First-order moment estimation of attenuation rate Second-order moment estimation of attenuation rate .

[0051] 12.2 Learning Rate Scheduling: A cosine annealing restart strategy is used. Total number of training rounds. Initial number of warm-up rounds The learning rate during the warm-up phase started from... linear growth to Then perform cosine annealing: ; in , One epoch constitutes one annealing cycle, and the process is reset at the end of each cycle. Then restart the annealing process.

[0052] Step 13: Online Difficult Example Mining In each training batch, all candidate anchor boxes (including positive and negative samples) are sorted according to the loss value. Sort the samples from highest to lowest loss. Only the top 30% of anchor boxes with the highest loss values ​​are selected for gradient backpropagation, while the remaining low-loss (easily classified) samples are not included in the parameter update of this iteration.

[0053] Step 14: Data Augmentation Strategy To improve the model's generalization ability, online data augmentation is performed on the training set images before each training round, using the following combined strategy: Random rotation: in to Rotate randomly within the range to maintain diversity in signature orientation.

[0054] Random scaling: The scaling factor is uniformly sampled within the interval.

[0055] Color jitter: Brightness adjustment factor Contrast factor Saturation factor .

[0056] Mosaic enhancement: Randomly select 4 images, scale them, and then stitch them together into one. The coordinates of the corresponding bounding boxes in the new image are transformed synchronously.

[0057] Cutout: A random occlusion in the image A square area is set to a pixel value of 0 to simulate a scenario where the signature is partially obscured.

[0058] MixUp: Mixes two images with a probability of 0.5 based on their weights. Pixel-level blending is performed, and labels are also blended with the same weight.

[0059] This invention employs a multi-task joint training strategy, combining Focal Loss to alleviate positive and negative sample imbalance, CIoULoss to improve localization accuracy, signature-aware edge loss to strengthen edge constraints, and center point offset loss to supervise structured information output. Through an online hard example mining mechanism, the top 30% of hard examples with the highest loss values ​​in each batch are selected for gradient backpropagation, forcing the model to focus on difficult samples (such as semi-transparent signatures highly similar to the background, or partially occluded signatures). Simultaneously, a combined data augmentation strategy and cosine annealing to restart the learning rate scheduling significantly improve the model's generalization ability. In cross-domain generalization experiments on the test set, the detection accuracy for unseen signature styles remains above 88%.

[0060] Step 15: Weighted Nonmaximum Suppression For the set of all predicted bounding boxes output by the detector head ,in For bounding box coordinates, The confidence score is... The system is categorized. Weighted NMS is used instead of traditional NMS to improve detection continuity in scenarios with overlapping signatures or occlusion.

[0061] 15.1 Clustering: Sort the boxes in descending order of confidence level, select the highest confidence box as the baseline, and then cluster the remaining boxes whose IoU with the baseline exceeds a threshold. The frames are grouped into the same cluster.

[0062] 15.2 Weighted Fusion: For K bounding boxes in the same cluster, the fused new coordinates are the weighted sum of the coordinates of each bounding box according to its confidence level. ; The confidence score after fusion is the highest confidence score within the cluster: .

[0063] 15.3 Output: Repeat the above steps until all bounding boxes have been processed, and output the final list of detection results. Step 16: Model Output and Result Storage For each test contract image, the model output includes: the detected seal type (official seal / financial seal / legal representative seal / sealing seal), and bounding box coordinates. Coordinates of the center point of the signature Confidence score All results are saved in JSON format, and bounding boxes and labels can be drawn on the image to generate a visual detection result map.

[0064] Step 17: Experimental Results and Comparative Analysis To verify the effectiveness of this invention, it was evaluated on a self-built contract signing test set (1,200 images), and changes in key metrics during training were recorded. Training used the same configuration as the preceding steps (batch size 16, optimizer AdamW, initial learning rate 1×10⁻⁶). -3 The cosine annealing restart strategy was used, with a total of 200 training epochs. During the training and validation process, the loss and evaluation metrics were recorded every 5 epochs, and the model with the highest mAP50-95 on the validation set was selected as the optimal model.

[0065] Definition of evaluation indicators train / box_loss: Bounding box localization loss on the training set, calculated using CIoU Loss, reflects the model's learning of the position, width, height, and aspect ratio of the signature bounding box.

[0066] train / cls_loss: Classification loss on the training set, calculated using Focal Loss, reflects the model's ability to distinguish between different types of signatures.

[0067] train / dfl_loss: Distribution focus loss on the training set, used to optimize the probability distribution estimation of bounding box coordinates and improve the refinement of bounding box regression.

[0068] precision(B): Precision on the validation set, defined as the proportion of samples that are actually positive out of all samples predicted as positive by the model. The formula is: ; TP represents a true positive (a correctly detected signature) and FP represents a false positive (a falsely detected non-signature area).

[0069] metrics / recall(B): Recall on the validation set, defined as the proportion of all real signatures correctly detected by the model, calculated using the following formula: ; FN represents a false negative (a signature that was missed during the inspection).

[0070] val / box_loss: Bounding box localization loss on the validation set, calculated using CIoU Loss, used to monitor the model's localization generalization ability on unseen data.

[0071] val / cls_loss: Classification loss on the validation set, calculated using Focal Loss, used to monitor the model's classification generalization ability on unseen data.

[0072] val / dfl_loss: Distribution focus loss on the validation set, used to monitor the model's ability to generalize bounding box distribution estimation on unseen data.

[0073] metrics / mAP50(B): The average accuracy on the validation set when the IoU threshold is 0.5, reflecting the overall detection performance of the model under relaxed localization conditions.

[0074] metrics / mAP50-95(B): The average accuracy of the model on the validation set with IoU thresholds from 0.5 to 0.95 and a step size of 0.05, reflecting the overall detection performance of the model under strict localization conditions.

[0075] The experimental comparison results are shown in Table 1: Table 1. Experimental comparison results of this invention with other models. ; Experimental results are as follows Figure 3 As shown, the detection results after interfering with the image are as follows: Figure 4 As shown, the method of this invention removes complex background interference through a document region segmentation network in the image preprocessing stage, and enhances the contrast of the signature area by combining CLAHE and gamma correction, effectively addressing challenges such as the complex visual characteristics of signatures (semi-transparency, fine texture, blurred edges, overlap with background text or patterns) and the variable shooting environment (uneven lighting, tilted angle, resolution differences). Through the collaborative work of the SAFE module, SGDA module, and multi-scale detection head, the mAP50 reaches 93.5% on the self-built test set, an improvement of 7.3 percentage points compared to the original YOLOv12, and the recall rate reaches 90.7%, significantly improving the robustness and accuracy of signature detection. Based on the improved YOLOv12 backbone network, this invention maintains a good balance between detection speed and accuracy through the lightweight design of the SAFE and SGDA modules, and has good engineering deployment value.

[0076] Based on the same inventive concept as the above-described method embodiments, this application also provides a contract signature detection system based on an improved YOLOv12. This system can achieve the functions provided by the above-described method embodiments, such as... Figure 5 As shown, the system includes the following modules: Image preprocessing module: Taking contract document images (such as scanned copies or photos taken with a mobile phone) as input, the first step is to normalize the size. An anti-aliased bicubic interpolation algorithm is used to uniformly scale the image to 640×640 pixels and convert it to the RGB color space. To further eliminate interference from complex backgrounds (such as desktop textures, handheld devices, shadows, etc.) on signature detection, a lightweight document region segmentation network is introduced. A U-Net architecture with MobileNetV3-Small as the encoder is used to predict binary masks for document regions. Training is performed using a weighted sum of binary cross-entropy loss and soft IoU loss. After morphological closing operations to fill the mask holes, the original image is multiplied element-wise with the mask to obtain the document foreground image. To enhance the model's robustness to different lighting and contrast levels, the document foreground image is sequentially subjected to contrast-limited adaptive histogram equalization and gamma correction (γ=1.2), effectively improving dark details and suppressing uneven lighting, resulting in a preprocessed enhanced image. The preprocessed enhanced image is then input into an improved YOLOv12 detection network.

[0077] The signature-aware feature enhancement module introduces a fine-grained signature-aware feature enhancement module (SAFE) into the YOLOv12 backbone network. Each SAFE module contains three parallel deformable convolutional layers (v3) with different dilation rates (3×3 kernels, dilation rate d∈{1,2,3}). The deformable convolution learns a two-dimensional offset and modulation scalar for each sampling point, enabling the receptive field to adaptively fit the irregular edges and curved textures of the signature, effectively addressing the semi-transparent nature and deformation issues of the signature. The output feature maps of the three branches are concatenated along the channel dimension, and then feature fusion and channel dimensionality reduction are performed through 1×1 convolution. This module adaptively samples the edge texture of the signature area through deformable convolution, while combining multi-branch dilated convolution to expand the receptive field and capture multi-scale contextual information of the signature. The feature map output by the feature enhancement module enhances the response to semi-transparent areas and blurred boundaries of the signature while preserving the original semantic information, and is added to the original feature map through residual connections to ensure smooth gradient flow.

[0078] The signature-guided attention module embeds a signature-guided channel-spatial hybrid attention (SGDA) mechanism between the backbone network and the detection head. This mechanism cascades channel attention and spatial attention, and introduces residual connections to achieve deep fusion of original and enhanced features. The channel attention submodule first performs global average pooling and global max pooling on the input feature map, obtaining two feature vectors, which are then fed into a shared two-layer fully connected network (compression ratio r=16). The two output channel attention vectors are added and activated by a sigmoid function to generate channel attention weights. These weights are then multiplied element-wise with the original feature map to filter feature channels related to the signature and suppress interference from background text and textures. The spatial attention submodule performs average pooling and max pooling on the channel-recalibrated feature map along the channel dimension. The resulting 2D feature maps are concatenated and then processed through a 7×7 convolutional layer and activated by a sigmoid function to generate spatial attention weights. These weights are then multiplied element-wise with the channel-enhanced feature map to generate a saliency heatmap of the signature foreground, guiding the model to focus on potential signature regions. The output feature map of the attention module is further residually connected with the original feature map to ensure lossless information transfer.

[0079] Multi-scale detection head improvement and optimization module: Addressing the uneven distribution of signature sizes, the YOLOv12 detection head is optimized. In addition to the original three detection scales, a new detection branch is added for extremely small signatures. Its input is the high-resolution feature map output from the second stage of the backbone network, used for accurate detection of signatures smaller than 16×16 pixels. Simultaneously, an improved BiFPN structure is adopted, introducing learnable weights into the top-down and bottom-up bidirectional paths to weightedly fuse features from different scales, improving the expressive power of multi-scale features. The size and proportion of anchor boxes are redesigned. The K-Means++ algorithm is used to cluster the labeled statistical information of signatures in the training dataset, generating 12 prior anchor boxes that match the signature shape, which are assigned to the four detection branches. The detection head outputs the class confidence, bounding box regression parameters, and the offset of the signature center point for each anchor box, providing richer structured information for subsequent signature comparison and verification.

[0080] Training and optimization module: An end-to-end multi-task joint training strategy is adopted, and a four-part loss function is designed, including classification loss, localization loss, signature perception loss, and center point offset loss. During training, a combined data augmentation strategy is used in conjunction with the high-quality signature dataset labeled with LabelMe to significantly improve the model's generalization ability. The optimizer is AdamW, with an initial learning rate set to 1×10⁻⁶. -3 Weight decay 5×10 -4In addition, the learning rate is adjusted in conjunction with the cosine annealing restart strategy. At the same time, the online hard example mining OHEM is enabled to focus on hard samples. In each batch, the top 30% of hard examples are selected according to the loss value for gradient backpropagation; the model is forced to focus on hard samples.

[0081] Post-processing module: The prediction results output by the detection head are subjected to weighted non-maximum suppression, and the IoU threshold is set to 0.5. For a group of overlapping boxes, the new coordinates after fusion are the weighted sum of the coordinates of each box according to its confidence score. The confidence score after fusion is the maximum confidence score within the cluster, which effectively improves the detection continuity in the case of overlapping or partial occlusion of signatures. Finally, the module outputs the position coordinates (boundary box coordinates), signature type (official seal / legal person seal / sealing seal), center point coordinates of signatures and confidence scores of all signatures in the contract image. The results are saved in JSON format, and a visual detection image is generated for easy manual review.

[0082] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0083] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0084] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0085] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0086] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A contract signature detection method based on an improved YOLOv12, characterized in that, Includes the following steps: S1. Image preprocessing: S2. Improved YOLOv12 detection network and training optimization; S3. Input the preprocessed enhanced image into the trained improved YOLOv12 detection network and output the prediction result; S4. Post-process the output prediction results.

2. The contract signature detection method based on improved YOLOv12 according to claim 1, characterized in that: Step S1 specifically includes the following: S11. Take the contract document image as input, first perform size normalization processing, use anti-aliased bicubic interpolation algorithm to uniformly scale the image to 640×640 pixels, and convert it to RGB color space; S12. A lightweight document region segmentation network is introduced. The U-Net architecture with MobileNetV3-Small as the encoder is used to predict the binary mask of the document region. The network is trained by weighted sum of binary cross-entropy loss and soft IoU loss. After filling the mask holes by morphological closing operation, the original image and the mask are multiplied element by element to obtain the document foreground image. S13. Perform contrast-limited adaptive histogram equalization and gamma correction sequentially on the foreground image of the document to obtain the preprocessed enhanced image.

3. The contract signature detection method based on improved YOLOv12 according to claim 1, characterized in that: In step S2, the improved YOLOv12 detection network specifically refers to the following: compared with the YOLOv12 backbone network, the improved YOLOv12 detection network further includes: a signature perception feature enhancement module, a signature-guided attention module, and a multi-scale detection head improvement and optimization module.

4. The contract signature detection method based on improved YOLOv12 according to claim 3, characterized in that: The signature-aware feature enhancement module specifically includes: introducing a fine-grained signature-aware feature enhancement module SAFE into the YOLOv12 backbone network. Each SAFE module contains three parallel deformable convolutions v3 with different dilation rates. The deformable convolutions learn a two-dimensional offset and modulation scalar for each sampling point, enabling the receptive field to adaptively fit the irregular edges and curved textures of the signature. The output feature maps of the three branches are concatenated in the channel dimension, and then feature fusion and channel dimensionality reduction are performed through 1×1 convolution to output an enhanced feature map, which is then added to the original feature map through residual connections.

5. The contract signature detection method based on improved YOLOv12 according to claim 3, characterized in that: The signature-guided attention module includes: embedding a signature-guided channel-spatial hybrid attention (SGDA) mechanism between the backbone network and the detection head. This mechanism cascades channel attention and spatial attention, and introduces residual connections to achieve deep fusion of original features and enhanced features; specifically, it includes the following: S51, Channel Attention: First, the input feature map... F Simultaneously, global average pooling (GAP) and global max pooling (GMP) are performed to obtain two feature vectors, which are then fed into a shared two-layer fully connected network. The output of the two channel attention vectors is summed and activated by a sigmoid function to generate channel attention weights. ,Will and F Element-wise multiplication yields the channel recalibration features. ; S52, Spatial Attention: (This appears to be a fragment of a sentence and doesn't translate directly.) Average pooling and max pooling are performed along the channel dimension. The two generated 2D feature maps are then concatenated and spatial attention weights are generated by passing them through a 7×7 convolutional layer and sigmoid activation. ,Will and Element-wise multiplication yields spatially enhanced features. ; S53, Residual Linkage: [This part is incomplete and likely refers to a separate section or code.] F and Residual connections, output feature maps .

6. The contract signature detection method based on improved YOLOv12 according to claim 3, characterized in that: The improved and optimized multi-scale detection head module includes: adding a detection branch for extremely small signatures based on the original three detection scales; specifically including the following: S61. Improve and optimize the detection head of YOLOv12. On the basis of the original three detection scales, add a new detection branch for extremely small signatures, which is used to accurately detect extremely small signatures with an area of ​​less than 16×16 pixels. S62. An improved BiFPN structure is adopted, which introduces learnable weights in the bidirectional paths of top-down and bottom-up to perform weighted fusion of features at different scales and improve the expressive power of multi-scale features. S63. Redesign the size and proportion of the anchor boxes. Use the K-Means++ algorithm to cluster the statistical information of the signatures in the training dataset to generate 12 prior anchor boxes that match the shape of the signatures, and assign them to the 4 detection branches respectively. S64. The detection head outputs the category confidence of each anchor point box, the bounding box regression parameters, and the offset of the signature center point.

7. The contract signature detection method based on improved YOLOv12 according to claim 1, characterized in that: The training optimization specifically includes: employing an end-to-end multi-task joint training strategy and designing a four-part loss function, including classification loss, localization loss, signature perception loss, and center point offset loss. During training, a combined data augmentation strategy is used in conjunction with the high-quality signature dataset annotated by LabelMe, and a cosine annealing restart strategy is employed to adjust the learning rate. Simultaneously, the online hard example mining OHEM is enabled to focus on difficult samples, and the top 30% of hard examples in each batch, sorted by loss value, are selected for gradient backpropagation.

8. The contract signature detection method based on improved YOLOv12 according to claim 1, characterized in that: The post-processing of the output prediction results specifically includes: applying weighted non-maximum suppression to the prediction results output by the detection head, setting the IoU threshold to 0.5; for a group of overlapping boxes, the fused new coordinates are the weighted sum of the coordinates of each box according to its confidence score, and the confidence score after fusion is the maximum confidence score within the cluster, which effectively improves the detection continuity in cases of overlapping or partial occlusion of signatures; finally, the output includes the position coordinates of all signatures in the contract image, the signature category, the coordinates of the signature center point, and the confidence score, and the results are saved in JSON format, while a visual detection image is generated.

9. A system for contract signature detection based on an improved YOLOv12 according to any one of claims 1-8, characterized in that: Includes the following modules: Image preprocessing module: Taking the contract document image as input, it first performs size normalization, then uses an anti-aliased bicubic interpolation algorithm to uniformly scale the image to 640×640 pixels and converts it to RGB color space; a lightweight document region segmentation network is introduced, using a U-Net architecture with MobileNetV3-Small as the encoder to predict binary masks for document regions. It is trained by a weighted sum of binary cross-entropy loss and soft IoU loss. After morphological closing operation to fill the mask holes, the original image and the mask are multiplied element-wise to obtain the document foreground image; the document foreground image is then subjected to contrast-limited adaptive histogram equalization and gamma correction to obtain the preprocessed enhanced image; The signature-aware feature enhancement module introduces a fine-grained signature-aware feature enhancement module (SAFE) into the YOLOv12 backbone network. Each SAFE module contains three parallel deformable convolutions (v3) with different dilation rates. The deformable convolutions learn a two-dimensional offset and modulation scalar for each sampling point, enabling the receptive field to adaptively fit the irregular edges and curved textures of the signature. The output feature maps of the three branches are concatenated along the channel dimension, and then feature fusion and channel dimensionality reduction are performed through 1×1 convolutions to output an enhanced feature map. This enhanced feature map is then added to the original feature map through residual connections. Signature-guided attention module: A signature-guided channel-spatial hybrid attention (SGDA) mechanism is embedded between the backbone network and the detection head. This mechanism cascades channel attention and spatial attention and introduces residual connections to achieve deep fusion of original features and enhanced features. Multi-scale detection head improvement and optimization module: Based on the original three detection scales, a detection branch for extremely small signatures is added to accurately detect extremely small signatures with an area of ​​less than 16×16 pixels. An improved BiFPN structure is adopted, introducing learnable weights in the bidirectional paths of top-down and bottom-up to perform weighted fusion of features at different scales, thereby improving the expressive power of multi-scale features. The size and proportion of the anchor boxes are redesigned, and the K-Means++ algorithm is used to cluster the labeled statistical information of the signatures in the training dataset to generate 12 prior anchor boxes that match the shape of the signatures, which are assigned to the four detection branches respectively. The detection head outputs the class confidence, bounding box regression parameters, and offset of the signature center point for each anchor box. Training optimization module: It adopts an end-to-end multi-task joint training strategy and designs a four-part loss function, which includes classification loss, localization loss, signature perception loss and center point offset loss. During training, a combination of data augmentation strategies was adopted, along with a cosine annealing restart strategy, to adjust the learning rate using the high-quality signature dataset labeled with LabelMe. At the same time, the online hard example mining OHEM was enabled to focus on hard samples, and the top 30% of hard examples in each batch were selected by sorting by loss value for gradient backpropagation. Post-processing module: The prediction results output by the detection head are subjected to weighted non-maximum suppression, and the IoU threshold is set to 0.

5. For a group of overlapping boxes, the new coordinates after fusion are the weighted sum of the coordinates of each box according to its confidence score. The confidence score after fusion is the maximum confidence score within the cluster, which effectively improves the detection continuity in the case of overlapping or partially occluded signatures. Finally, the module outputs the position coordinates of all signatures in the contract image, the signature category, the coordinates of the signature center point, and the confidence score. The results are saved in JSON format, and a visual detection image is generated at the same time.

Images