Express signature image detection method, device and equipment and storage medium

Abstract

This invention relates to the field of intelligent logistics technology, and in particular to a method, apparatus, device, and storage medium for detecting express delivery receipt images. The method first acquires a set of receipt images, performs anomaly detection on the receipt images to obtain a sample set of receipts to be tested, then extracts multimodal features from the sample set of receipts to obtain a multimodal feature vector set, performs spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result, uses a dual-branch detection network to perform fine-grained tampering detection and localization on the multimodal feature vector set to obtain tampering results, then performs physical consistency verification on the multimodal feature vector set to obtain physical verification results, and finally integrates the scoring results, tampering results, and physical verification results to obtain a multidimensional detection dataset, and performs confidence weighted calculation on the multidimensional detection dataset to obtain anomaly detection results, aiming to improve the accuracy and interpretability of detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics technology, and in particular to a method, apparatus, equipment, and storage medium for detecting express delivery receipt images. Background Technology

[0002] With the rapid development of e-commerce and express logistics, the standardization and security of express delivery signing have become a key focus of the industry. Photo signing has become the mainstream form of performance verification. Couriers are required to take a photo of the package upon delivery and upload it to the system as a basis for service completion, liability definition, and dispute resolution.

[0003] Current express delivery signature image detection solutions still have significant technical shortcomings, making it difficult to adapt to the security and efficiency requirements of actual business operations. Most existing detection methods rely solely on single visual features or simple rules, resulting in a limited detection dimension and insufficient coverage of abnormal scenarios. Furthermore, these solutions are poorly robust against advanced forgery techniques such as splicing, photo manipulation, and deepfakes, easily leading to false positives and false negatives, and failing to effectively resist malicious forgery. In addition, existing detection models are mostly black-box judgments, lacking interpretable criteria and failing to clearly identify anomaly types and suspicious areas, hindering business-side traceability and anomaly handling. These issues directly affect the authenticity and reliability of signature receipts, restricting the level of security control in the express delivery signing process. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, apparatus, device and storage medium for detecting express delivery receipt images, which aims to improve the accuracy and interpretability of the detection and solve the problems of existing receipt image detection having a single dimension, poor robustness to advanced forgery and lack of basis for judgment.

[0005] The first aspect of this invention provides a method for detecting express delivery receipt images, comprising: acquiring a set of receipt image sets; performing anomaly detection on the receipt image set to obtain a sample set of receipts to be tested; extracting multimodal features from the sample set of receipts to be tested to obtain a multimodal feature vector set; performing spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result; employing a dual-branch detection network to perform fine-grained tampering detection and localization on the multimodal feature vector set to obtain a tampering result; performing physical consistency verification on the multimodal feature vector set to obtain a physical verification result; integrating the scoring result, the tampering result, and the physical verification result to obtain a multidimensional detection dataset, and performing confidence-weighted calculation on the multidimensional detection dataset to obtain anomaly detection results.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of performing anomaly detection on the signed receipt image set to obtain a sample set of signed receipts to be tested includes: performing lightweight preprocessing on the signed receipt image set using an image preprocessing tool to obtain a preprocessed image set; obtaining a pre-trained anomaly detection model, the anomaly detection model including a feature extraction module, an anomaly judgment module, and a sample screening module, the feature extraction module, the anomaly judgment module, and the sample screening module being connected in sequence; performing feature extraction processing on the preprocessed image set based on the feature extraction module to obtain a basic feature set; performing anomaly detection processing on the basic feature set based on the anomaly judgment module to obtain a preliminary anomaly judgment result set; and performing anomaly test sample screening processing on the preliminary anomaly judgment result set based on the sample screening module to obtain the sample set of signed receipts to be tested.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of extracting multimodal features from the sample set of signed receipts to be tested to obtain a multimodal feature vector set includes: acquiring the signature time-series dynamic data and spatiotemporal data corresponding to each signed receipt sample in the sample set of signed receipts to be tested; extracting visual features from the sample set of signed receipts to be tested using a visual encoder to obtain a visual feature vector set; extracting handwriting dynamic features from each of the signature time-series dynamic data using a handwriting dynamic encoder to obtain a handwriting dynamic feature vector set; extracting spatiotemporal features from each of the spatiotemporal data using a spatiotemporal consistency verification tool to obtain a spatiotemporal feature vector set; and integrating the visual feature vector set, the handwriting dynamic feature vector set, and the spatiotemporal feature vector set to obtain the multimodal feature vector set.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of performing spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result includes: obtaining a preset spatiotemporal logic verification rule and a spatiotemporal feature vector set in the multimodal feature vector set; constructing a spatiotemporal graph based on the spatiotemporal feature vector set; inputting the spatiotemporal graph into a pre-trained graph neural network model for neighborhood information aggregation processing to obtain a vertex feature set; and performing spatiotemporal consistency scoring on the vertex feature set based on the spatiotemporal logic verification rule to obtain the scoring result.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the dual-branch detection network includes a feature adaptation module, a dual-branch detection module, and a result fusion optimization module. The feature adaptation module, the dual-branch detection module, and the result fusion optimization module are connected sequentially. The dual-branch detection module further includes a global branch sub-module and a local branch sub-module, which are respectively connected to the feature adaptation module. The step of using the dual-branch detection network to perform fine-grained tamper detection and localization processing on the multimodal feature vector set to obtain the tamper result includes: inputting the multimodal feature vector set into the feature adaptation module for feature dimension alignment processing to obtain a fused feature map; inputting the fused feature map into the global branch sub-module for global authenticity determination processing to obtain a determination result; inputting the fused feature map into the local branch sub-module for pixel-level tamper segmentation processing to obtain a pixel-level tamper region mask; and inputting the determination result and the pixel-level tamper region mask into the result fusion optimization module for attention-guided optimization processing to obtain the tamper result.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the result fusion optimization module includes a preliminary fusion submodule, an attention guidance submodule, and an optimization submodule, which are sequentially connected. The step of inputting the judgment result and the pixel-level tampered region mask into the result fusion optimization module for attention guidance optimization processing to obtain the tampering result includes: inputting the judgment result and the pixel-level tampered region mask into the preliminary fusion submodule for preliminary fusion processing to obtain a preliminary tampering detection result; inputting the preliminary tampering detection result into the attention guidance submodule for gradient calculation processing to obtain an attention heatmap; and inputting the attention heatmap into the optimization submodule for correction processing to obtain the tampering result.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of performing physical consistency verification on the multimodal feature vector set to obtain a physical verification result includes: performing illumination consistency verification on the multimodal feature vector set using a light source estimation algorithm to obtain an illumination anomaly determination result; performing noise pattern consistency verification on the multimodal feature vector set using a noise distribution analysis algorithm to obtain a noise pattern consistency verification result; performing recompression trace detection on the multimodal feature vector set using an encoding parsing algorithm to obtain a recompression trace detection result; and integrating the illumination anomaly determination result, the noise pattern consistency verification result, and the recompression trace detection result to obtain the physical verification result.

[0012] A second aspect of the present invention provides a device for detecting express delivery receipt images, comprising: an anomaly detection module for acquiring a set of receipt images and performing anomaly detection on the receipt image set to obtain a sample set of receipts to be tested; a feature extraction module for extracting multimodal features from the sample set of receipts to be tested to obtain a multimodal feature vector set; a spatiotemporal scoring module for performing spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result; a tampering detection module for using a dual-branch detection network to perform fine-grained tampering detection and localization on the multimodal feature vector set to obtain a tampering result; a physical verification module for performing physical consistency verification on the multimodal feature vector set to obtain a physical verification result; and a result output module for integrating the scoring result, the tampering result, and the physical verification result to obtain a multidimensional detection dataset, and performing confidence weighted calculation on the multidimensional detection dataset to obtain an anomaly detection result.

[0013] A third aspect of the present invention provides a courier delivery receipt image detection device, the courier delivery receipt image detection device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the courier delivery receipt image detection device to perform each step of the courier delivery receipt image detection method described above.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the express delivery signature image detection method described in any of the preceding claims.

[0015] In the technical solution of this invention, a set of signed receipt images is first acquired, and anomaly detection is performed on the signed receipt image set to obtain a sample set of signed receipts to be tested. Then, multimodal feature extraction is performed on the sample set of signed receipts to be tested to obtain a multimodal feature vector set. Spatiotemporal consistency scoring processing is performed on the multimodal feature vector set to obtain a scoring result. A dual-branch detection network is used to perform fine-grained tampering detection and localization processing on the multimodal feature vector set to obtain tampering results. Next, physical consistency verification is performed on the multimodal feature vector set to obtain physical verification results. Finally, the scoring results, tampering results, and physical verification results are integrated to obtain a multidimensional detection dataset. Confidence weighted calculation is performed on the multidimensional detection dataset to obtain anomaly detection results. This aims to improve the accuracy and interpretability of detection and solve the problems of existing signed receipt image detection methods, such as single dimension, poor robustness against advanced forgery, and lack of basis for judgment. Attached Figure Description

[0016] Figure 1 This is a first flowchart of the express delivery signature image detection method provided in an embodiment of the present invention; Figure 2This is a second flowchart of the express delivery signature image detection method provided in an embodiment of the present invention; Figure 3 This is a third flowchart of the express delivery signature image detection method provided in the embodiments of the present invention; Figure 4 This is a fourth flowchart of the express delivery signature image detection method provided in this embodiment of the invention; Figure 5 This is a fifth flowchart of the express delivery signature image detection method provided in the embodiments of the present invention; Figure 6 This is a sixth flowchart of the express delivery signature image detection method provided in the embodiments of the present invention; Figure 7 This is the seventh flowchart of the express delivery signature image detection method provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the structure of the express delivery receipt image detection device provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of the express delivery receipt image detection device provided in an embodiment of the present invention. Detailed Implementation

[0017] This invention provides a method, apparatus, device, and storage medium for detecting express delivery receipt images. In this invention, the terms "first," "second," "third," "fourth," etc. (if applicable) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the express delivery signature image detection method in this invention includes: 101. Obtain a set of signed receipt images, perform anomaly detection on the signed receipt image set, and obtain a sample set of signed receipts to be tested; In this embodiment, the collected receipt image set is first subjected to lightweight preprocessing. Image preprocessing tools are used to reduce the amount of image data and computational complexity, providing support for subsequent rapid detection. Then, a pre-trained anomaly detection model is invoked. This model has been optimized through adversarial robustness enhancement training. During training, diverse real and generated samples are integrated. Different styles of signature samples are generated through the StyleGAN3 model, and image features after physical attacks such as screen re-photographing and printing scanning are simulated using the CycleGAN model. At the same time, adversarial perturbations such as FGSM and PGD are added to generate adversarial samples to ensure that the model has strong anti-interference capabilities. The anomaly detection model comprises a feature extraction module, an anomaly judgment module, and a sample screening module connected in sequence. The feature extraction module extracts basic features from the preprocessed image set. The anomaly judgment module makes a preliminary anomaly judgment based on the basic features. The sample screening module selects suspected anomaly samples from the preliminary judgment results, ultimately forming a sample set of signed receipts to be tested. This avoids wasting computational resources on ineffective detection and provides accurate targets for subsequent deep detection steps such as multimodal feature extraction and fine-grained tamper detection, ensuring the efficiency and accuracy of the overall detection process and enhancing the reliability and security of express delivery signed receipt image detection.

[0019] 102. Perform multimodal feature extraction on the sample set of the signed receipts to be tested to obtain a multimodal feature vector set; In this embodiment, the signature time-series dynamic data and spatiotemporal data corresponding to each sample of the signed receipt to be tested are first collected to provide complete data support for multi-dimensional feature extraction. A visual encoder is used to extract visual features from each sample of the signed receipt to be tested. The sample image is first preprocessed by standardization, size normalization and noise removal. Then, it is divided into blocks according to different scales and candidate boxes for key areas of interest such as signature, date, and seal are divided. The block image is input into the Swin Transformer encoder to extract global semantic features of each region and capture the correlation information between regions. Then, the output feature map is input into a multi-layer CNN network. Through multi-scale convolution operation, the subtle texture features of key regions are extracted. The global semantic features and local texture features are fused and the dimensions are aligned to generate a visual feature vector. The visual feature vectors of all samples are integrated to form a visual feature vector set. A handwriting dynamic encoder is used to process the signature temporal dynamic data of each sample. First, the temporal signals such as pressure change curve, writing speed, and pen stroke turning are cleaned and normalized to remove outliers and noise. Then, they are input into an LSTM network step by step. A bidirectional LSTM captures the positive and negative dependencies of the temporal data to generate an initial temporal feature sequence. An attention mechanism is introduced to strengthen the feature representation of key time steps. After integration, a handwriting dynamic feature vector is generated and summarized to form a handwriting dynamic feature vector set. A spatiotemporal consistency verification tool is used to extract spatiotemporal features from the spatiotemporal data of each sample. A spatiotemporal graph is constructed with the signing time, GPS trajectory points, and delivery personnel path nodes as vertices and adjacent GPS points and continuous path nodes as edges. After assigning corresponding attributes to vertices and edges, the graph is input into a GNN model. The graph convolutional layer aggregates vertex neighborhood information and updates feature representations to capture the spatiotemporal correlation between nodes, extract spatiotemporal features, and form a spatiotemporal feature vector set. Finally, the visual feature vector set, the handwriting dynamic feature vector set, and the spatiotemporal feature vector set are integrated to obtain a multimodal feature vector set. This set provides high-quality feature support for subsequent steps such as spatiotemporal consistency scoring, fine-grained tamper detection, and physical consistency verification, significantly improving the accuracy, robustness, and reliability of the overall detection solution. It effectively addresses various forgery and abnormal scenarios and meets the actual business needs of express delivery signature image detection.

[0020] 103. Perform spatiotemporal consistency scoring on the multimodal feature vector set to obtain the scoring result; In this embodiment, a preset spatiotemporal logic verification rule is first obtained. Simultaneously, a spatiotemporal feature vector set is extracted from the multimodal feature vector set. This vector set includes the delivery time, GPS track points, delivery person path nodes, and their corresponding attribute information, providing basic data support for subsequent processing. A spatiotemporal graph is constructed based on the extracted spatiotemporal feature vector set. The delivery time, GPS track points, and delivery person path nodes are used as vertices of the spatiotemporal graph, and adjacent GPS points and consecutive nodes of the delivery path are used as edges. Attributes such as timestamps, latitude and longitude, and path order are assigned to vertices, while attributes such as time intervals and spatial distances are assigned to edges, thus completing the construction of the spatiotemporal graph. The constructed spatiotemporal graph is input into a pre-trained graph neural network model. The model's graph convolutional layers aggregate the neighborhood information of each vertex, updating the vertex feature representation, thereby accurately capturing the spatiotemporal relationships between vertices and obtaining a vertex feature set. Based on preset spatiotemporal logic verification rules, spatiotemporal consistency scoring is performed on the vertex feature set. By calculating core verification indicators such as the matching degree between the signing time and GPS trajectory time, the spatial continuity of the delivery person's path, and the rationality of the time interval between nodes, and combining the weight allocation of each indicator, the spatiotemporal consistency score corresponding to each multimodal feature vector is output. All scoring data are integrated to form the final scoring result, which can provide reliable spatiotemporal dimension scoring support for the subsequent fusion of multidimensional detection results and confidence weighted calculation, further improving the anomaly identification capability and detection accuracy of the overall detection scheme, strengthening the rigor of express delivery signing image detection, and effectively avoiding detection misjudgments caused by spatiotemporal logic anomalies.

[0021] 104. A dual-branch detection network is used to perform fine-grained tamper detection and localization processing on the multimodal feature vector set to obtain the tampering result; In this embodiment, the dual-branch detection network is composed of a feature adaptation module, a dual-branch detection module, and a result fusion optimization module connected sequentially. The dual-branch detection module includes a global branch sub-module and a local branch sub-module set set in parallel. First, the multimodal feature vector set is input into the feature adaptation module to complete the feature dimension alignment processing and generate a fused feature map adapted for subsequent detection. Then, the fused feature map is simultaneously input into the global branch sub-module and the local branch sub-module. The global branch sub-module extracts global image features through multiple convolutional and fully connected layers, and the classifier completes the overall authenticity judgment and outputs the judgment result. The local branch sub-module relies on the U-Net++ architecture, extracts multi-scale semantic features through encoder downsampling, and the decoder combines upsampling and skip connections to fuse multi-level features and restore spatial resolution. The segmentation head achieves pixel-level classification and outputs a pixel-level tampered region mask. Subsequently, the judgment result is fused with the pixel-level tampered region mask input result through an attention-guided optimization module. By calculating the gradient of the feature map of each layer of the model, the Grad-CAM++ algorithm is used to perform gradient weighted summation to generate an attention heatmap. The heatmap is then superimposed on the original feature map image to highlight the suspected tampered locations. Finally, the global judgment, region mask, and highlighted location information are integrated to form a complete tampering result. Through global and local dual-branch collaborative detection, both the overall image authenticity judgment and pixel-level tampering location are considered, achieving fine-grained anomaly identification. The attention-guided mechanism can strengthen the feature representation of key regions, improve the accuracy and visualization of tampering location, and feature dimension alignment ensures the effective adaptation of multimodal features, effectively overcoming the limitations of a single detection branch, enhancing the detection capability for complex forgery scenarios, providing accurate evidence of suspicious regions for subsequent physical consistency verification, and comprehensively improving the reliability and effectiveness of tampering detection.

[0022] 105. Perform physical consistency verification on the multimodal feature vector set to obtain the physical verification results; In this embodiment, a light source estimation algorithm is first used to verify illumination consistency. Features related to object edges, shadow boundaries, and specular reflection areas are extracted from the multimodal feature vector set. Different object contours, such as signatures, seals, and backgrounds, are marked. Based on the direction, length, and intensity of shadow boundaries, combined with the location of specular reflection highlight areas, the light source direction and intensity parameters corresponding to each object are calculated. By comparing the light source parameters of different objects, the physical logic is verified, and it is determined whether there are any illumination inconsistencies, thus obtaining the illumination anomaly judgment result. Subsequently, a noise distribution analysis algorithm is used to verify noise pattern consistency. The image corresponding to the multimodal feature vector set is subjected to high-pass filtering to extract high-frequency noise residuals and remove low-frequency image subject information. The noise residual image is divided into multiple local regions, and the noise mean, variance, frequency distribution, and other distribution characteristics of each region are calculated. By comparing the noise distribution characteristics of different local regions, regions with abrupt changes in noise distribution are identified, and it is determined whether this is caused by splicing images from different sources. The noise pattern consistency verification result is then output. Finally, an encoding parsing algorithm is used for recompression trace detection. The JPEG encoding information of the image is parsed, and the quantization table and DCT coefficient distribution data are extracted. The numerical characteristics of the quantization table are analyzed and compared with the quantization table patterns of normal single-compression images. The proportions of DC and AC components and the distribution patterns of absolute values ​​of the DCT coefficients are statistically analyzed to determine whether they conform to the characteristics of secondary compression. The recompression trace detection results are obtained by combining the two types of analysis results. After completing the verification in three dimensions, the results of illumination anomaly judgment, noise pattern consistency verification, and recompression trace detection are integrated to form a complete physical verification result. Verification is carried out from three physical dimensions: illumination, noise, and encoding, comprehensively covering the core features of image forgery, making up for the limitations of single-dimensional verification, and accurately identifying various forgery scenarios such as synthetic images, spliced ​​images, and secondary compressed images. Each verification algorithm is highly targeted and can effectively capture subtle physical feature anomalies, improving the accuracy and reliability of physical consistency verification and reducing false positives caused by forged images.

[0023] 106. Integrate the scoring results, the tampering results, and the physical verification results to obtain a multi-dimensional detection dataset, and perform confidence weighted calculation on the multi-dimensional detection dataset to obtain anomaly detection results.

[0024] In this embodiment, scoring results, tampering results, and physical verification-related data are first integrated to construct a complete multi-dimensional detection dataset. Combined with relevant information obtained from semantic understanding, the confidence parameters for each detection dimension are clarified. The confidence weighting calculation requires first determining the weight allocation for each detection dimension. The weight coefficients are then adjusted based on the semantic understanding results. During the calculation process, the confidence of each detection dimension is first standardized to clarify the weight proportion of different detection dimensions. Then, a weighted summation method is used to multiply the confidence of each dimension's detection result by its corresponding weight and sum them to obtain a preliminary weighted result. Subsequently, the preliminary weighted result is corrected using semantic understanding-related data to remove abnormal data and ensure that the calculation result conforms to physical logic and semantic specifications. After completing the weighted calculation, the results of each detection dimension are integrated to form an anomaly detection result that includes anomaly type, confidence score, suspicious area annotation, and semantic analysis conclusions. This result covers the confidence parameters of each detection dimension, anomaly type annotation, and suspicious area information. Simultaneously, blockchain-based evidence-based data is incorporated to ensure the traceability of the result. The anomaly detection results obtained through this process can be directly used for subsequent blockchain evidence storage and traceability, providing operators with clear judgment criteria. At the same time, it supports the display of results and data verification on the business side, improves the reliability and rigor of the overall detection system, ensures that the detection results can accurately reflect the actual situation, meet the detection needs in business scenarios, provide complete detection basis for subsequent blockchain evidence storage and data traceability, and ensure the stability and practicality of the detection results.

[0025] Please see Figure 2 In this embodiment of the invention, the step of performing anomaly detection on the signed receipt image set to obtain a sample set of signed receipts to be tested includes: 201. Use an image preprocessing tool to perform lightweight preprocessing on the signed receipt image set to obtain a preprocessed image set; 202. Obtain a pre-trained anomaly detection model, wherein the anomaly detection model includes a feature extraction module, an anomaly judgment module, and a sample screening module, wherein the feature extraction module, the anomaly judgment module, and the sample screening module are connected in sequence; 203. Based on the feature extraction module, perform feature extraction processing on the preprocessed image set to obtain a basic feature set; 204. Based on the anomaly judgment module, perform anomaly detection processing on the basic feature set to obtain a preliminary anomaly judgment result set; 205. Based on the sample screening module, perform abnormal test sample screening on the preliminary abnormal judgment result set to obtain the test receipt sample set.

[0026] In this embodiment, lightweight image processing tools such as OpenCV and PIL can be used for image preprocessing. These tools can efficiently perform core preprocessing operations such as size compression and color space conversion, effectively reducing the amount of image data and computational complexity, adapting to the needs of subsequent rapid detection, and providing standardized input data for the anomaly detection model. The acquired pre-trained anomaly detection model is based on the lightweight LeYOLO model, integrating a feature extraction module, anomaly detection module, and sample selection module. These three modules are connected sequentially to form a complete logical architecture of feature extraction, anomaly detection, and sample selection. At the same time, adversarial robustness enhancement design is incorporated, combining StyleGAN3, CycleGAN models, and adversarial perturbation generation mechanisms to improve the model's resistance to interference from various forgery and attack scenarios. The training process of this anomaly detection model requires first constructing a sample dataset, collecting real sample data such as signatures, dates, and seals of different styles, as well as real sample data after screen capture and printing scans. Then, the StyleGAN3 model is used to generate forged signature samples covering different writing styles, stroke thickness, and texture features. An image style transfer network is constructed using the CycleGAN model to convert real receipt images into image features after simulating physical attacks. The generated samples and real samples are integrated to expand the training dataset and to complete the labeling of forgery types and attack methods. During training, adversarial perturbations of different intensities generated by methods such as FGSM and PGD are added to the training samples. The original training samples and adversarial samples are mixed and input into the model for training. During backpropagation, the detection accuracy of the model on the two types of samples is optimized simultaneously. The strength and generation method of the adversarial perturbations are iteratively adjusted to balance the detection performance of the model on normal samples and adversarial samples, avoiding the model from overfitting to the adversarial perturbations. After training, the adversarial robustness of the model is verified using an independent test set. The model parameters are adjusted until the preset robustness index is reached, and the model pre-training is completed.

[0027] In this embodiment, after the model is deployed, the basic features of key regions are extracted from the preprocessed image set by the feature extraction module to obtain the basic feature set. Then, the anomaly judgment module performs rapid anomaly detection on the basic feature set to identify whether there are suspected anomalies in the image and obtain the preliminary anomaly judgment result set. Finally, the sample screening module screens the preliminary anomaly judgment result set, selects samples with suspected anomalies as the sample set of the signed receipt to be tested, and removes samples with obvious no anomalies to reduce the computational load of subsequent depth detection.

[0028] Please see Figure 3 In this embodiment of the invention, the step of extracting multimodal features from the sample set of signed receipts to be tested to obtain a multimodal feature vector set includes: 301. Obtain the signature timing dynamic data and spatiotemporal data corresponding to each signature sample in the sample set of the signed receipts to be tested; 302. Visual features are extracted from the sample set of the signed receipts to be tested using a visual encoder to obtain a set of visual feature vectors; 303. Use a handwriting dynamic encoder to extract handwriting dynamic features from each of the signature time-series dynamic data to obtain a handwriting dynamic feature vector set; 304. Use a spatiotemporal consistency verification tool to extract spatiotemporal features from each of the spatiotemporal data to obtain a spatiotemporal feature vector set; 305. Integrate the visual feature vector set, the handwriting dynamic feature vector set, and the spatiotemporal feature vector set to obtain the multimodal feature vector set.

[0029] In this embodiment, the signature timing dynamic data is a continuous timing signal generated during the signing process, covering pressure change curves, writing speed, pen strokes, etc., which can truly reflect the dynamic characteristics of the signature behavior and is used to distinguish between real writing and imitation, and to generate handwriting; the spatiotemporal data is time and space correlation data related to the signing behavior, including signing time, GPS track points, delivery personnel path nodes, and information such as timestamps, latitude and longitude, path order, time intervals between adjacent nodes, and spatial distances corresponding to these nodes, which is used to mine the spatiotemporal logical correlation of the signing behavior.

[0030] In this embodiment, the visual encoder is a dedicated encoding model for extracting visual features from the signed receipt image. Its core consists of a preprocessing module, a Swing Transformer encoder, and a multi-layer CNN network, enabling image preprocessing, multi-scale feature extraction and fusion, and accurate capture of global and local image features. The handwriting dynamic encoder is an encoding model for extracting temporal dynamic features of the signature. It uses an LSTM network as its core and combines an attention mechanism to capture the dependencies and key dynamic features of the signature temporal data. The spatiotemporal consistency verification tool is a tool for extracting spatiotemporal data features and performing spatiotemporal correlation analysis. It integrates a GNN model and spatiotemporal logic verification rules to capture spatiotemporal correlation features.

[0031] In this embodiment, the signature time-series dynamic data and spatiotemporal data corresponding to each sample of the signed receipt to be tested are first obtained to provide complete data support for multimodal feature extraction. A visual encoder is used to extract visual features from each sample of the signed receipt to be tested. First, the sample image is standardized, its size is normalized and noise is removed through its preprocessing module. Then, it is divided into blocks according to different scales and candidate boxes for key areas of interest such as signature, date and seal are divided. The block image is input into the SwinTransformer encoder, and the global semantic features of each region are extracted and the inter-region correlation information is captured through the window attention mechanism. Then, the output feature map is input into a multi-layer CNN network, and multi-scale convolution operations are performed through convolutional layers and pooling layers to extract the subtle texture features of key areas. Finally, through feature concatenation and dimension alignment, the global semantic features and local texture features are fused to generate a visual feature vector. The visual feature vectors of all samples are integrated to form a visual feature vector set. Then, a handwriting dynamic encoder is used to process the signature time-series dynamic data of each sample. First, the time-series signal is cleaned and normalized to remove outliers and noise. Then, the preprocessed time-series data is input into the LSTM network step by step. The forward LSTM and backward LSTM capture the forward and backward dependencies of the time-series data respectively to generate the initial time-series feature sequence. An attention mechanism is introduced to calculate the attention weight of the feature at each time step, which strengthens the feature representation of key time steps such as pressure change, speed change, and pen stroke turning. After integration, a handwriting dynamic feature vector is generated and summarized to form a handwriting dynamic feature vector set. Finally, a spatiotemporal consistency verification tool was used to extract spatiotemporal features from the spatiotemporal data of each sample. A spatiotemporal graph was constructed with the delivery time, GPS trajectory points, and delivery personnel path nodes as vertices, and adjacent GPS points and continuous nodes of the delivery path as edges. Vertices were assigned attributes such as timestamps, latitude and longitude, and path order, while edges were assigned attributes such as time intervals and spatial distances. The constructed spatiotemporal graph was input into a built-in GNN model, and the neighborhood information of each vertex was aggregated through graph convolutional layers to update the vertex feature representations, capture the spatiotemporal relationships between nodes, extract spatiotemporal features, and form a spatiotemporal feature vector set. Finally, the visual feature vector set, the handwriting dynamic feature vector set, and the spatiotemporal feature vector set were integrated to obtain a multimodal feature vector set.

[0032] In this embodiment, the visual encoder achieves efficient fusion of global semantics and local texture features of the image, capturing detailed information in key areas such as signatures and seals; the handwriting dynamic encoder can deeply mine the dynamic features of the signing process, effectively distinguishing between genuine handwriting and forged handwriting; the spatiotemporal consistency verification tool can fully capture the spatiotemporal correlation of signing behavior and identify potential spatiotemporal logical anomalies. The complementary fusion of these three types of features compensates for the limitations of a single feature dimension, significantly improving the comprehensiveness and accuracy of feature representation, filtering out invalid noise interference, and retaining core feature information. The generated multimodal feature vector set can provide high-quality feature support for subsequent deep detection steps such as spatiotemporal consistency scoring, fine-grained tamper detection, and physical consistency verification, further improving the accuracy, robustness, and reliability of the overall detection scheme, effectively dealing with various forgery and abnormal scenarios, meeting the needs of express delivery signing image detection for multi-dimensional and high-precision features, and laying a solid foundation for the efficient advancement of the overall detection process.

[0033] Please see Figure 4 In this embodiment of the invention, the step of performing spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result includes: 401. Obtain the preset spatiotemporal logic verification rules and the spatiotemporal feature vector set in the multimodal feature vector set; 402. Construct a spatiotemporal graph based on the aforementioned spatiotemporal feature vector set; 403. Input the spatiotemporal graph into a pre-trained graph neural network model for neighborhood information aggregation processing to obtain a vertex feature set; 404. Based on the spatiotemporal logic verification rules, perform spatiotemporal consistency scoring on the vertex feature set to obtain the scoring result.

[0034] In this embodiment, it is first necessary to clarify the specific meaning of the spatiotemporal logic verification rules. These rules are standardized judgment criteria set based on the actual operational logic of express delivery business, the spatiotemporal correlation patterns of normal signing behavior, and the statistical analysis results of historical real signing samples. They cover the matching degree threshold between signing time and GPS trajectory time, the spatial continuity threshold of delivery personnel's path, and the reasonableness threshold of time interval between nodes. They also include judgment criteria for spatiotemporal correlation anomalies, which are used to quantitatively evaluate the logical consistency of each spatiotemporal element and provide a clear basis for subsequent scoring.

[0035] In this embodiment, after obtaining the preset spatiotemporal logic verification rules, a spatiotemporal feature vector set is extracted from the multimodal feature vector set. This vector set includes the receipt time, GPS track points, delivery personnel path nodes, and attribute information such as timestamps, latitude and longitude, and path order corresponding to each node, as well as correlation information such as time intervals and spatial distances between adjacent GPS points and continuous path nodes, providing complete data support for the construction of the spatiotemporal map. The construction process of the spatiotemporal map is based on the spatiotemporal feature vector set. First, the receipt time, GPS track points, and delivery personnel path nodes are used as vertices of the spatiotemporal map, and each vertex is assigned corresponding attributes such as timestamps, latitude and longitude, and path order. Then, GPS points at adjacent times and continuous nodes of the delivery path are used as edges of the spatiotemporal map, and each edge is assigned corresponding attributes such as time intervals and spatial distances. Through the association and combination of vertices and edges, discrete spatiotemporal data is transformed into a structured spatiotemporal map, realizing the visualization and correlation of spatiotemporal information.

[0036] In this embodiment, the constructed spatiotemporal graph is input into a pre-trained graph neural network model for neighborhood information aggregation processing. The training architecture of this model is based on a graph convolutional layer, integrating a feature aggregation layer, an activation layer, and an output layer. The graph convolutional layer is responsible for extracting and aggregating vertex neighborhood information, the feature aggregation layer is used to integrate neighborhood information and vertex features, the activation layer realizes nonlinear feature transformation, and the output layer is used to output the aggregated vertex features. The overall architecture is adapted to the structured features of the spatiotemporal graph and can efficiently capture the spatiotemporal relationships between nodes. The training process of the model first requires building a training dataset, collecting real spatiotemporal data of delivery receipts and simulated spatiotemporal anomaly data, and generating diverse spatiotemporal samples using models such as StyleGAN3, covering samples from different delivery scenarios and different spatiotemporal correlation patterns. After labeling the samples, the spatiotemporal graph samples are input into the model for training. During training, the neighborhood information of each vertex is aggregated and processed through graph convolutional layers to update the feature representation of the vertex, capture the spatiotemporal correlation between nodes, optimize model parameters simultaneously, and iteratively adjust the training strategy to avoid model overfitting. After training, an independent test set is used to verify the performance of the model, focusing on verifying the model's accuracy in capturing spatiotemporal correlation and its ability to identify anomalies. The model parameters are adjusted until the preset performance indicators are achieved, completing the model pre-training.

[0037] In this embodiment, the graph neural network model processes the vertex feature set, and performs spatiotemporal consistency scoring on the vertex feature set based on preset spatiotemporal logic verification rules. Specifically, it calculates three core indicators: the matching degree between the delivery time and the GPS trajectory time, the spatial continuity of the delivery person's path, and the rationality of the time interval between nodes. Combining the weight allocation of each indicator, it quantifies and scores each indicator, and then obtains the final spatiotemporal consistency score by weighted summation. The final scoring result includes multiple aspects, including a comprehensive spatiotemporal consistency score, specific quantitative values ​​of the three core verification indicators, and annotations of spatiotemporal logic anomaly judgment results. It clarifies whether spatiotemporal travel anomalies exist and their types, and also includes details such as the location of anomaly nodes and the specific deviations of anomaly indicators. The multi-dimensional information covered in the scoring result not only clearly reflects the spatiotemporal logic rationality of the signing behavior, but also provides high-quality spatiotemporal dimension support for the subsequent integration of multi-dimensional detection datasets and confidence weighted calculation. This compensates for the limitations of single-dimensional detection, further improves the accuracy and rigor of the overall detection solution, effectively avoids detection misjudgments caused by spatiotemporal logic anomalies, and meets the core requirement of spatiotemporal dimension verification for express delivery signing image detection.

[0038] Please see Figure 5 In this embodiment of the invention, the dual-branch detection network includes a feature adaptation module, a dual-branch detection module, and a result fusion optimization module. The feature adaptation module, the dual-branch detection module, and the result fusion optimization module are connected sequentially. The dual-branch detection module further includes a global branch sub-module and a local branch sub-module, which are respectively connected to the feature adaptation module. The step of using the dual-branch detection network to perform fine-grained tamper detection and localization processing on the multimodal feature vector set to obtain the tamper result includes: 501. Input the multimodal feature vector set into the feature adaptation module for feature dimension alignment processing to obtain a fused feature map; 502. Input the fused feature map into the global branch submodule for global authenticity determination to obtain the determination result; 503. Input the fused feature map into the local branch submodule for pixel-level tampering segmentation processing to obtain a pixel-level tampering region mask; 504. Input the determination result and the pixel-level tampered region mask into the result fusion optimization module for attention-guided optimization processing to obtain the tampering result.

[0039] In this embodiment, a dual-branch detection network is used to perform fine-grained tamper detection and localization processing on the multimodal feature vector set to obtain the tamper results. The network is composed of a feature adaptation module, a dual-branch detection module, and a result fusion optimization module connected in sequence. The dual-branch detection module includes a global branch sub-module and a local branch sub-module set in parallel. The modules work together to realize a complete detection process from feature adaptation to tamper result output.

[0040] In this embodiment, the multimodal feature vector set is first input into the feature adaptation module. Through feature dimension alignment processing, the dimensional differences between different modal features are eliminated, and multi-dimensional features such as visual, handwriting dynamics, and spatiotemporal features are integrated into a unified fusion feature map. This ensures that the features can adapt to the input requirements of the subsequent dual-branch detection module, laying the foundation for accurate detection. Subsequently, the fusion feature map is simultaneously input into the global branch submodule and the local branch submodule. The global branch submodule extracts global semantic features from the fusion feature map through multiple convolutional layers and fully connected layers, capturing the overall feature association of the image. This is then input into the classifier to complete the overall image authenticity judgment and obtain the judgment result. The local branch submodule adopts the U-Net++ architecture. The encoder part performs downsampling operations on the fusion feature map to gradually extract multi-scale semantic features and capture image information at different levels. The decoder part performs deep fusion of features at different levels of the encoder through upsampling and skip connections, gradually restoring the spatial resolution of the image and achieving pixel-level feature reconstruction. A segmentation head is added to the decoder output to classify each pixel, accurately marking the tampered area and the non-tampered area, and obtaining a pixel-level tampered area mask. Finally, the judgment result and the pixel-level tampered region mask input result are fused and optimized by the module to perform attention-guided optimization processing. By calculating the gradient of the feature maps of each layer of the dual-branch detection network with respect to the detection result, regions that contribute highly to the authenticity judgment and tamper location are captured. The Grad-CAM++ algorithm is used to perform weighted summation of the gradients to generate an attention heatmap. The heatmap is superimposed on the original feature map image to highlight the suspicious tampering locations. By integrating the judgment result, pixel-level tampered region mask, attention heatmap and related detection details, the final tampering result is obtained. The obtained tampering result contains multi-dimensional core information that can fully support the subsequent detection process, specifically including the overall image judgment result and clarifying the image... The system includes: a tampering detection function and its overall confidence level; a pixel-level tampering area mask to accurately mark the specific location, range, and pixel distribution of the tampered area, achieving pixel-level localization of the tampered area; an attention heatmap to visualize the key areas of focus during the model detection process, highlighting suspicious tampering locations for easy and intuitive identification of key tampering points; a tampering type determination result, clearly identifying the tampering method as splicing, forgery, or re-photographing, as well as a quantitative assessment of the degree of tampering; and relevant explanations of the detection process, covering brief descriptions of key steps such as feature extraction, gradient calculation, and heatmap generation. This provides accurate evidence of suspicious areas for subsequent physical consistency verification and supports the traceability and verification of detection results.

[0041] In this embodiment, the training process of the dual-branch detection network must follow a systematic training strategy to ensure that the model has high-precision tamper detection and localization capabilities. First, a training dataset is constructed, collecting real receipt image samples of different styles and scenarios, covering normal samples and various tampered samples, including splicing, re-photographing, and signature forgery. All samples are accurately labeled, indicating the tamper type, location of the tampered area, and degree of tampering, providing supervision for model training. During training, the labeled samples are input into the dual-branch detection network, simultaneously training the feature adaptation module, dual-branch detection module, and result fusion optimization module. At the same time, adversarial perturbations of different intensities generated by methods such as FGSM and PGD are added to the training samples to generate adversarial samples. The original training samples and adversarial samples are mixed and input into the model. During backpropagation, the model's detection accuracy for both original and adversarial samples is simultaneously optimized. The model parameters and the strength and generation method of the adversarial perturbations are iteratively adjusted to balance the model's detection performance on normal and adversarial samples, avoiding overfitting of the model to adversarial perturbations or a single tampering type. During training, the focus is on optimizing the feature extraction capability of the dual-branch detection module and the attention guidance effect of the result fusion optimization module to ensure the accuracy of the global branch sub-module's authenticity determination and the pixel-level localization accuracy of the local branch sub-module. After training, an independent test set is used to verify the model performance, focusing on verifying the tamper detection accuracy, tamper region localization accuracy, and adversarial robustness. The model parameters are adjusted until the preset performance indicators are achieved, and the model training is completed.

[0042] Please see Figure 6 In this embodiment of the invention, the result fusion optimization module includes a preliminary fusion submodule, an attention guidance submodule, and an optimization submodule, which are sequentially connected. The step of inputting the judgment result and the pixel-level tampered region mask into the result fusion optimization module for attention guidance optimization processing to obtain the tampered result includes: 601. Input the determination result and the pixel-level tampering region mask into the preliminary fusion submodule for preliminary fusion processing to obtain the preliminary tampering detection result; 602. Input the preliminary tampering detection results into the attention guidance submodule for gradient calculation processing to obtain an attention heatmap; 603. Input the attention heatmap into the optimization submodule for correction processing to obtain the tampering result.

[0043] In this embodiment, the result fusion and optimization module consists of a preliminary fusion submodule, an attention-guided submodule, and an optimization submodule connected sequentially. The judgment result and pixel-level tampering region mask are input into the preliminary fusion submodule for preliminary fusion processing. This integrates the overall image authenticity judgment information with pixel-level tampering region location data, eliminates invalid and redundant information, filters out key areas suspected of tampering, clarifies the preliminary characteristics and scope of tampering, and forms preliminary tampering detection results. This provides targeted basic data support for subsequent attention-guided optimization, ensuring the optimization process focuses on the detection target. Subsequently, the preliminary tampering detection results are input into the attention-guided submodule for gradient calculation processing. Specifically, based on the preliminary tampering detection results, the gradient of each layer of the dual-branch detection network's feature map to this result is calculated. Regions in the feature map that contribute significantly to authenticity judgment and tampering location are captured. The Grad-CAM++ algorithm is then used to perform a weighted summation of the calculated gradients, generating an attention heatmap. This heatmap visualizes the key image regions during the detection process, clearly presenting the core areas of suspected tampering, achieving visualized location of key tampering areas, and solving the problem of ambiguous tampering region location in traditional detection. Finally, the attention heatmap is input into the optimization submodule for correction. Combining the judgment information from the initial tampering detection results with the pixel-level tampering region mask, the attention heatmap is overlaid with the original feature mapping image to highlight suspicious tampering locations, further correcting the positioning deviation of the tampering region, eliminating false suspicious regions, and strengthening the feature representation of the real tampering region. At the same time, the judgment results, pixel-level tampering region mask, attention heatmap, and related detection details are integrated to form a complete and accurate tampering result. The generated tampering result can not only accurately present whether tampering has occurred, the location of the tampered region, and the focus of the tampering, but also provide clear evidence of suspicious regions for subsequent physical consistency verification. It also facilitates operators to intuitively identify tampering details and conduct traceability verification, significantly improving the practicality and rigor of fine-grained tampering detection, adapting to the actual business needs of express delivery signature image tampering detection, and strengthening the effectiveness of signature certificate authenticity control.

[0044] Please see Figure 7 In this embodiment of the invention, the step of performing physical consistency verification on the multimodal feature vector set to obtain physical verification results includes: 701. The illumination consistency of the multimodal feature vector set is verified by using a light source estimation algorithm to obtain the illumination anomaly determination result; 702. The noise distribution analysis algorithm is used to verify the noise mode consistency of the multimodal feature vector set, and the noise mode consistency verification result is obtained. 703. The multimodal feature vector set is subjected to recompression trace detection using an encoding parsing algorithm to obtain the recompression trace detection result; 704. Integrate the illumination anomaly determination result, the noise pattern consistency verification result, and the recompression trace detection result to obtain the physical verification result.

[0045] In this embodiment, the light source estimation algorithm can specifically employ a light source estimation algorithm based on shadow geometry or an LS3D light source estimation algorithm. These algorithms can accurately capture the feature correlation between shadows and specular reflections in the image and adapt to the visual feature information in the multimodal feature vector set. The noise distribution analysis algorithm can specifically employ a noise distribution analysis algorithm combining high-pass filtering and wavelet transform, which can efficiently extract high-frequency noise residuals and analyze their distribution patterns. The encoding parsing algorithm can specifically employ a JPEG encoding parsing algorithm combined with DCT coefficient statistical analysis, which can accurately parse image encoding information and identify recompression traces.

[0046] In the specific verification process, a light source estimation algorithm is first used to verify illumination consistency. Features related to object edges, shadow boundaries, and specular reflection areas are extracted from the multimodal feature vector set. Different object contours, such as signatures, seals, and backgrounds, are marked. Based on the direction, length, and intensity of shadow boundaries, combined with the location of specular reflection highlight areas, the algorithm calculates the light source direction and intensity parameters corresponding to each object. The physical logic of the light source parameters of different objects is verified by comparing them, determining whether there are any illumination inconsistencies, and thus obtaining the illumination anomaly judgment result. If the difference in light source parameters between different objects exceeds a preset threshold, it is determined to be a synthetic image. Subsequently, a noise distribution analysis algorithm is used to verify noise pattern consistency. The algorithm performs high-pass filtering on the image corresponding to the multimodal feature vector set, extracting high-frequency noise residuals and removing low-frequency image subject information. The noise residual image is divided into multiple local regions, and the distribution characteristics such as the noise mean, variance, and frequency distribution of each region are calculated. By comparing the noise distribution characteristics of different local regions, regions with abrupt changes in noise distribution are identified, determining whether they are caused by splicing images from different sources. The noise pattern consistency verification result is output, and the area and location of the regions with abrupt changes in noise distribution are statistically analyzed. Finally, a coding parsing algorithm is used for recompression trace detection. The algorithm parses the JPEG encoding information of the image, extracts the quantization table and DCT coefficient distribution data, analyzes the numerical characteristics of the quantization table and compares it with the quantization table patterns of normal single-compression images, and statistically analyzes the proportion of DC and AC components and the distribution patterns of absolute values ​​of the DCT coefficients to determine whether they conform to the characteristics of double compression. The recompression trace detection results are obtained by combining the two types of analysis results. After completing the verification in three dimensions, the results of illumination anomaly judgment, noise pattern consistency verification, and recompression trace detection are integrated to form a complete physical verification result. This comprehensively reflects the consistency of the image at the physical feature level, achieving a deep integration of physical features and multi-dimensional features, reducing false positives caused by forged images. The generated physical verification result provides reliable physical dimension support for the subsequent integration of multi-dimensional detection datasets and confidence weighted calculations, further improving the overall detection system, strengthening the resistance to advanced forgery methods, ensuring the rigor of express delivery receipt image detection, meeting the logistics industry's needs for verifying the authenticity of receipts, and providing a solid guarantee for the accuracy of the overall detection results.

[0047] The above describes the express delivery signature image detection method in the embodiments of the present invention. The following describes the express delivery signature image detection device in the embodiments of the present invention. Please refer to [link / reference]. Figure 8 One embodiment of the express delivery signature image detection device of the present invention includes: Anomaly detection module 801: used to acquire a set of signed receipt images, perform anomaly detection on the set of signed receipt images, and obtain a sample set of signed receipts to be tested; Feature extraction module 802: used to perform multimodal feature extraction on the sample set of the signed receipts to be tested, and obtain a multimodal feature vector set; Spatiotemporal scoring module 803: used to perform spatiotemporal consistency scoring on the multimodal feature vector set to obtain a scoring result; Tampering detection module 804: used to perform fine-grained tampering detection and localization processing on the multimodal feature vector set using a dual-branch detection network to obtain tampering results; Physical verification module 805: used to perform physical consistency verification on the multimodal feature vector set and obtain physical verification results; Result output module 806: used to integrate the scoring result, the tampering result and the physical verification result to obtain a multi-dimensional detection dataset, and to perform confidence weighted calculation on the multi-dimensional detection dataset to obtain anomaly detection results.

[0048] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0049] above Figure 8 The express delivery receipt image detection device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The express delivery receipt image detection device in this embodiment of the invention is described in detail below from the perspective of hardware processing.

[0050] Figure 9 This is a schematic diagram of the structure of a parcel delivery receipt image detection device 900 provided in an embodiment of the present invention. The parcel delivery receipt image detection device 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the parcel delivery receipt image detection device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the parcel delivery receipt image detection device 900 to implement the steps of the parcel delivery receipt image detection method provided in the above-described method embodiments.

[0051] The express delivery signature image detection device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 9 The structure of the express delivery receipt image detection device shown does not constitute a limitation on the express delivery receipt image detection device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the express delivery signature image detection method.

[0053] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting a delivery signature image, the method comprising: receiving a delivery image; and detecting a signature image in the delivery image. include: Obtain a set of signed receipt images, perform anomaly detection on the signed receipt image set, and obtain a sample set of signed receipts to be tested; Multimodal feature extraction is performed on the sample set of the signed receipts to be tested to obtain a multimodal feature vector set; The spatiotemporal consistency scoring process is performed on the multimodal feature vector set to obtain the scoring result; A dual-branch detection network is used to perform fine-grained tamper detection and localization processing on the multimodal feature vector set to obtain the tampering result; Physical consistency verification is performed on the multimodal feature vector set to obtain the physical verification result; By integrating the scoring results, the tampering results, and the physical verification results, a multi-dimensional detection dataset is obtained. Confidence weighting is then performed on the multi-dimensional detection dataset to obtain anomaly detection results.

2. The express delivery signature image detection method according to claim 1, characterized in that, The step of performing anomaly detection on the signed receipt image set to obtain a sample set of signed receipts to be tested includes: The signed receipt image set was lightly preprocessed using an image preprocessing tool to obtain a preprocessed image set. A pre-trained anomaly detection model is obtained, the anomaly detection model including a feature extraction module, an anomaly judgment module and a sample screening module, the feature extraction module, the anomaly judgment module and the sample screening module being connected in sequence; Based on the feature extraction module, feature extraction processing is performed on the preprocessed image set to obtain a basic feature set; Based on the anomaly detection module, the basic feature set is processed to obtain a preliminary anomaly judgment result set. Based on the sample screening module, the abnormal preliminary judgment result set is processed to screen abnormal test samples to obtain the test receipt sample set.

3. The express delivery signature image detection method according to claim 1, characterized in that, The step of extracting multimodal features from the sample set of signed receipts to be tested, resulting in a multimodal feature vector set, includes: Obtain the signature time-series dynamic data and spatiotemporal data corresponding to each signature sample in the sample set of the signed receipts to be tested; A visual encoder is used to extract visual features from the sample set of signed receipts to be tested, resulting in a set of visual feature vectors. A handwriting dynamic encoder is used to extract handwriting dynamic features from each of the signature time-series dynamic data to obtain a handwriting dynamic feature vector set; A spatiotemporal consistency verification tool is used to extract spatiotemporal features from each spatiotemporal data to obtain a spatiotemporal feature vector set. The visual feature vector set, the handwriting dynamic feature vector set, and the spatiotemporal feature vector set are integrated to obtain the multimodal feature vector set.

4. The express delivery signature image detection method according to claim 3, characterized in that, The process of performing spatiotemporal consistency scoring on the multimodal feature vector set to obtain the scoring result includes: Obtain the preset spatiotemporal logic verification rules and the spatiotemporal feature vector set in the multimodal feature vector set; Construct a spatiotemporal graph based on the spatiotemporal feature vector set; The spatiotemporal graph is input into a pre-trained graph neural network model for neighborhood information aggregation processing to obtain a vertex feature set; The spatiotemporal consistency scoring process is performed on the vertex feature set based on the spatiotemporal logic verification rules to obtain the scoring result.

5. The express delivery signature image detection method according to claim 1, characterized in that, The dual-branch detection network includes a feature adaptation module, a dual-branch detection module, and a result fusion optimization module. The feature adaptation module, the dual-branch detection module, and the result fusion optimization module are connected in sequence. The dual-branch detection module also includes a global branch sub-module and a local branch sub-module. The global branch sub-module and the local branch sub-module are respectively connected to the feature adaptation module. The step of using a dual-branch detection network to perform fine-grained tamper detection and localization on the multimodal feature vector set to obtain tampering results includes: The multimodal feature vector set is input into the feature adaptation module for feature dimension alignment processing to obtain a fused feature map; The fused feature map is input into the global branch submodule for global authenticity determination to obtain the determination result; The fused feature map is input into the local branch submodule for pixel-level tampering segmentation processing to obtain a pixel-level tampering region mask. The determination result and the pixel-level tampered region mask are input into the result fusion optimization module for attention-guided optimization processing to obtain the tampering result.

6. The express delivery signature image detection method according to claim 5, characterized in that, The result fusion and optimization module includes a preliminary fusion submodule, an attention guidance submodule, and an optimization submodule, which are connected sequentially. The step of inputting the determination result and the pixel-level tampered region mask into the result fusion optimization module for attention-guided optimization processing to obtain the tampering result includes: The determination result and the pixel-level tampered region mask are input into the preliminary fusion submodule for preliminary fusion processing to obtain the preliminary tampering detection result; The preliminary tampering detection results are input into the attention guidance submodule for gradient calculation processing to obtain an attention heatmap. The attention heatmap is input into the optimization submodule for correction processing to obtain the tampering result.

7. The express delivery signature image detection method according to claim 1, characterized in that, The physical consistency verification of the multimodal feature vector set, to obtain the physical verification result, includes: The illumination consistency of the multimodal feature vector set is verified by using a light source estimation algorithm to obtain the illumination anomaly determination result; The noise distribution analysis algorithm is used to verify the noise mode consistency of the multimodal feature vector set, and the noise mode consistency verification result is obtained. The multimodal feature vector set is subjected to recompression trace detection using an encoding parsing algorithm to obtain the recompression trace detection result; The physical verification result is obtained by integrating the illumination anomaly determination result, the noise pattern consistency verification result, and the recompression trace detection result.

8. A device for detecting images of express delivery receipts, characterized in that, include: Anomaly detection module: used to acquire a set of signed receipt images, perform anomaly detection on the signed receipt image set, and obtain a sample set of signed receipts to be tested; Feature extraction module: used to extract multimodal features from the sample set of the signed receipts to be tested, and obtain a multimodal feature vector set; Spatiotemporal scoring module: used to perform spatiotemporal consistency scoring on the multimodal feature vector set to obtain the scoring result; Tampering detection module: used to perform fine-grained tampering detection and localization processing on the multimodal feature vector set using a dual-branch detection network to obtain tampering results; Physical verification module: used to perform physical consistency verification on the multimodal feature vector set and obtain physical verification results; The result output module is used to integrate the scoring results, the tampering results, and the physical verification results to obtain a multi-dimensional detection dataset, and to perform confidence weighted calculation on the multi-dimensional detection dataset to obtain anomaly detection results.

9. A device for detecting images of express delivery signatures, characterized in that, The express delivery receipt image detection device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the express delivery receipt image detection device to perform the various steps of the express delivery receipt image detection method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement each step of the express delivery signature image detection method as described in any one of claims 1-7.

Images