Signed photo watermark identification method, device and equipment and storage medium

Abstract

The invention relates to the technical field of image recognition processing, in particular to a sign-in photo watermark recognition method and device, equipment and a storage medium. The sign-in photo watermark identification method comprises the following steps: obtaining a plurality of photo samples shot when a courier signs in a door-to-door manner, and labeling watermark characters in the photo samples to obtain a labeled data set; constructing a GOT-OCR2 model as a basic model for watermark character recognition, and introducing an interactive OCR module into the basic model; training the basic model by adopting the annotation data set subjected to data enhancement processing, and calculating the loss of the prediction sequence and the annotation sequence by adopting a CTC loss function to obtain a watermark character recognition model; a watermark character recognition model is adopted to recognize a sign-in photo uploaded in real time, and recognition information is compared with actual sign-in information in a logistics system. According to the method, the watermark in the signed photo can be effectively identified, so that the efficiency of identifying the signed photo is effectively improved, and the uploading of a false signed photo is avoided.

Description

Technical Field

[0001] This invention relates to the field of image recognition and processing technology, and in particular to a method, apparatus, device, and storage medium for watermark recognition of signed receipt photos. Background Technology

[0002] To resolve disputes over liability during the express delivery signing process and ensure service transparency, the express delivery industry currently widely adopts door-to-door signing photo verification technology. This technology preserves visual evidence of the moment of signing, theoretically allowing for both compliance with courier operating procedures and protection of consumers' right to know. However, this technology system has a significant functional shortcoming: it severely lacks the ability to identify and verify key anti-counterfeiting information watermarks in the photos. This vulnerability directly provides fertile ground for fraudulent photo-taking.

[0003] This has led some delivery drivers to falsify their records to avoid performance evaluations for late deliveries and efficient door-to-door pickups. In the event of disputes such as damaged or lost packages, these falsified photos cannot serve as valid evidence for claims, further exacerbating consumer conflicts and severely damaging consumer trust in delivery services. Clearly, existing technology still needs improvement and enhancement. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method, apparatus, device and storage medium for recognizing watermarks on signed photos, in order to solve the technical problem that the system in the prior art cannot recognize image watermarks.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a method for watermark recognition in signed delivery photos, comprising the following steps: acquiring multiple photo samples taken by couriers when signing for packages, annotating the watermark text in the photo samples to obtain an annotated dataset; constructing a GOT-OCR2 model as the basic model for watermark text recognition, and introducing an interactive OCR module into the basic model; performing data augmentation processing on the annotated dataset, training the basic model using the data augmented annotated dataset, and calculating the loss between the predicted sequence and the annotated sequence using the CTC loss function to obtain a watermark text recognition model; using the watermark text recognition model to recognize the signed delivery photos uploaded in real time, outputting recognition information, comparing the recognition information with the actual signed delivery information in the logistics system, and outputting the comparison result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of acquiring multiple photo samples taken by couriers when signing for packages, and annotating the watermark text in the photo samples to obtain an annotated dataset, specifically includes: acquiring multiple photo samples taken by couriers when signing for packages, drawing bounding boxes for the watermark text in each photo, and recording the pixel coordinates of the upper left and lower right corners of the bounding boxes; annotating the font features of the watermark text, wherein the font features include font type, font size, font weight, and color; and annotating the format rules of the watermark content to clarify the composition structure of each field, thereby obtaining an annotated dataset.

[0007] Optionally, in the second implementation of the first aspect of the present invention, the construction of the GOT-OCR2 model as the basic model for watermark text recognition, and the introduction of an interactive OCR module into the basic model, specifically includes: constructing a high-compression encoder, wherein the high-compression encoder includes an input layer, a convolutional layer module, and a first Transformer encoder module arranged sequentially, the convolutional layer module including multiple convolutional layers, each convolutional layer being followed by a BatchNorm and ReLU activation function, and the first Transformer encoder module consisting of a multi-head self-attention mechanism and a feedforward neural network; constructing a long context decoder, wherein the long context decoder includes a second Transformer encoder module and an output layer arranged sequentially, the second Transformer encoder module including a multi-head self-attention mechanism and an encoder-decoder attention mechanism, and the output layer employing a linear mapping and a Softmax function; constructing the GOT-OCR2 model based on the high-compression encoder and the long context decoder, and introducing an interactive OCR module into the GOT-OCR2 model.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of constructing a GOT-OCR2 model based on a high-compression encoder and a long context decoder, and introducing an interactive OCR module into the GOT-OCR2 model, specifically includes: constructing a GOT-OCR2 model based on a high-compression encoder and a long context decoder as a base model; constructing a coordinate-guided region-level recognition submodule to generate a feature map based on a region mask; constructing a color-guided region-level recognition submodule to generate a feature map based on a color mask; constructing a fusion submodule to perform a weighted summation of the feature map based on the region mask and the feature map based on the color mask to obtain a watermark feature sequence; integrating the coordinate-guided region-level recognition submodule, the color-guided region-level recognition submodule, and the fusion submodule to obtain an interactive OCR module, and introducing the interactive OCR module into the GOT-OCR2 model.

[0009] Optionally, in the fourth implementation of the first aspect of the present invention, the step of performing data augmentation processing on the labeled dataset, training the base model using the data-augmented labeled dataset, and calculating the loss between the predicted sequence and the labeled sequence using the CTC loss function to obtain the watermark text recognition model specifically includes: performing data augmentation processing on the labeled dataset, wherein the data augmentation processing includes rotating, scaling, adjusting the watermark text, adjusting the color, adjusting the image lighting parameters, adding noise, and blurring the image; inputting the data-augmented labeled dataset into the base model to obtain the predicted character sequence; calculating the loss between the predicted sequence and the labeled sequence using the CTC loss function, and updating the model parameters using the Adam optimizer through the backpropagation algorithm to obtain the watermark text recognition model.

[0010] Optionally, in the fifth implementation of the first aspect of the present invention, the step of using a watermark text recognition model to recognize the real-time uploaded receipt photos, outputting recognition information, and comparing the recognition information with the actual receipt information in the logistics system to output a comparison result specifically includes: acquiring the real-time uploaded receipt photos, adjusting the size of the receipt photos, and performing grayscale normalization processing; using a watermark text recognition model to recognize the real-time uploaded receipt photos and outputting recognition information; extracting key information from the recognition information and comparing it with the actual receipt information in the logistics system to obtain a comparison result, wherein the key information includes the estimated receipt time, receipt location, and system record number; if any field in the comparison result does not match, the system automatically triggers a warning signal.

[0011] Optionally, in the sixth implementation of the first aspect of the present invention, the key information extracted from the identification information is compared with the actual receipt information in the logistics system to obtain a comparison result. The key information includes the estimated receipt time, receipt location, and system record number. Specifically, this includes: comparing the key information extracted from the identification information with the actual receipt information in the logistics system; checking whether the identified estimated receipt time string conforms to a preset format and whether the difference between it and the actual time is within an allowable range; calculating the similarity between the identified receipt location and the actual location using a string similarity algorithm, and determining a discrepancy if it is lower than a set threshold; and performing a complete string match between the identified system record number and the system record number in the logistics system, and determining a discrepancy if they do not match.

[0012] A second aspect of the present invention provides a watermark recognition device for signed delivery photos, comprising: an annotation module for acquiring multiple photo samples taken by couriers when signing for delivery, and annotating the watermark text in the photo samples to obtain an annotation dataset; a construction module for constructing a GOT-OCR2 model as the basic model for watermark text recognition, and introducing an interactive OCR module into the basic model; a training module for performing data augmentation processing on the annotation dataset, training the basic model using the data augmentation-processed annotation dataset, and calculating the loss between the predicted sequence and the annotation sequence using the CTC loss function to obtain a watermark text recognition model; and a recognition module for recognizing the signed delivery photos uploaded in real time using the watermark text recognition model, outputting recognition information, comparing the recognition information with the actual signed delivery information in the logistics system, and outputting the comparison result.

[0013] Optionally, in the first implementation of the second aspect of the present invention, the annotation module includes: a boundary annotation unit, used to acquire multiple photo samples taken by couriers when signing for packages, draw bounding boxes for the watermark text in each photo, and record the pixel coordinates of the upper left and lower right corners of the bounding boxes; a font annotation unit, used to annotate the font features of the watermark text, the font features including font type, font size, font weight, and color; and a format annotation unit, used to annotate the format rules of the watermark content to clarify the composition structure of each field in order to obtain an annotation dataset.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the construction module includes: a first construction unit for constructing a high-compression encoder, the high-compression encoder including an input layer, a convolutional layer module, and a first Transformer encoder module arranged sequentially, the convolutional layer module including multiple convolutional layers, each convolutional layer being followed by a BatchNorm and ReLU activation function, the first Transformer encoder module consisting of a multi-head self-attention mechanism and a feedforward neural network; a second construction unit for constructing a long context decoder, the long context decoder including a second Transformer encoder module and an output layer arranged sequentially, the second Transformer encoder module including a multi-head self-attention mechanism and an encoder-decoder attention mechanism, the output layer employing a linear mapping and a Softmax function; and a third construction unit for constructing a GOT-OCR2 model based on the high-compression encoder and the long context decoder, and introducing an interactive OCR module into the GOT-OCR2 model.

[0015] Optionally, in a third implementation of the second aspect of the present invention, the third construction unit includes: a model subunit, used to construct a GOT-OCR2 model based on a high-compression encoder and a long context decoder as a base model; a construction subunit, used to construct a coordinate-guided region-level recognition submodule to generate a feature map based on a region mask, construct a color-guided region-level recognition submodule to generate a feature map based on a color mask, construct a fusion submodule to perform a weighted summation of the feature map based on the region mask and the feature map based on the color mask to obtain a watermark feature sequence; and an integration subunit, used to integrate the coordinate-guided region-level recognition submodule, the color-guided region-level recognition submodule, and the fusion submodule to obtain an interactive OCR module, and to introduce the interactive OCR module into the GOT-OCR2 model.

[0016] Optionally, in a fourth implementation of the second aspect of the present invention, the training module includes: a data augmentation unit for performing data augmentation processing on the labeled dataset, the data augmentation processing including image rotation, scaling, watermark text adjustment, color adjustment, image lighting parameter adjustment, noise addition, and image blurring; a first training unit for inputting the data-augmented labeled dataset into a base model to obtain a predicted character sequence; and a second training unit for calculating the loss between the predicted sequence and the labeled sequence using the CTC loss function, and updating the model parameters using the Adam optimizer and backpropagation algorithm to obtain a watermark text recognition model.

[0017] Optionally, in the fifth implementation of the second aspect of the present invention, the identification module includes: a preprocessing unit, used to acquire real-time uploaded receipt photos, adjust the size of the receipt photos, and perform grayscale normalization processing; an identification unit, used to identify the real-time uploaded receipt photos using a watermark text recognition model and output identification information; a comparison unit, used to extract key information from the identification information and compare it with the actual receipt information in the logistics system to obtain a comparison result, wherein the key information includes the expected receipt time, receipt location, and system record number; and an early warning unit, used to automatically trigger an early warning signal if any field in the comparison result does not match.

[0018] Optionally, in a sixth implementation of the second aspect of the present invention, the comparison unit includes: an extraction subunit, used to extract key information from the identification information and compare it with the actual receipt information in the logistics system, wherein the key information includes the estimated receipt time, receipt location, and system record number; a first comparison subunit, used to check whether the identified estimated receipt time string conforms to a preset format and whether the difference between it and the actual time is within an allowable range; a second comparison subunit, used to calculate the similarity between the identified receipt location and the actual location using a string similarity algorithm, and if it is lower than a set threshold, it is determined to be inconsistent; and a third comparison subunit, used to perform a complete string match between the identified system record number and the system record number in the logistics system, and if they are inconsistent, it is determined to be inconsistent.

[0019] A third aspect of the present invention provides a watermark recognition device for signed receipt photos, including a memory and at least one processor, wherein the memory stores computer-readable instructions; the at least one processor invokes the computer-readable instructions in the memory to execute the various steps of the watermark recognition method for signed receipt photos as described above.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the various steps of the signature photo watermark recognition method described above.

[0021] Beneficial Effects: This invention provides a method for watermark recognition in signed delivery photos. The method first acquires multiple photo samples taken by couriers during door-to-door delivery signing, and annotates the watermark text in these samples to obtain a high-quality labeled dataset suitable for model training. Then, a GOT-OCR2 model is constructed as the base model for watermark text recognition, specifically for recognizing watermark text in photos taken by couriers during door-to-door delivery signing. An interactive OCR module is introduced into the base model to enhance its accuracy and robustness in recognizing watermark text. Next, before training, the labeled dataset undergoes data augmentation processing, and the base model is trained using this augmented dataset. The CTC loss function is used to calculate the loss between the predicted sequence and the labeled sequence to improve the watermark text recognition model's ability to recognize photos in different scenarios. Finally, the established watermark text recognition model is used to recognize real-time uploaded signed delivery photos, outputting accurate watermark text recognition information. This information is then compared with the actual signed delivery information in the logistics system, outputting the comparison result. This effectively prevents fraudulent photography, ensuring consumers receive genuine signed delivery information and thus improving their experience. Attached Figure Description

[0022] Figure 1This is a first flowchart of the watermark recognition method for signed receipt photos provided in an embodiment of the present invention; Figure 2 This is a second flowchart of the watermark recognition method for signed receipt photos provided in an embodiment of the present invention; Figure 3 This is a third flowchart of the watermark recognition method for signed receipt photos provided in an embodiment of the present invention; Figure 4 This is a fourth flowchart of the watermark recognition method for signed receipt photos provided in this embodiment of the invention; Figure 5 The fifth flowchart of the signature photo watermark recognition method provided in the embodiments of the present invention; Figure 6 The sixth flowchart of the signature photo watermark recognition method provided in the embodiments of the present invention; Figure 7 The seventh flowchart of the signature photo watermark recognition method provided in the embodiments of the present invention; Figure 8 A schematic diagram of a signature photo watermark recognition device provided in an embodiment of the present invention; Figure 9 Another structural schematic diagram of the signature photo watermark recognition device provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of the signature photo watermark recognition device provided in an embodiment of the present invention. Detailed Implementation

[0023] This invention provides a method, apparatus, device, and storage medium for watermark recognition in signed delivery photos. First, the invention collects a large number of photo samples taken by couriers during door-to-door delivery signing, and performs detailed annotation on the watermark text contained in the samples. This constructs a high-quality annotated dataset suitable for model training, laying a data foundation for subsequent recognition accuracy. At the model construction level, the GOT-OCR2 model is selected as the basic framework for watermark text recognition. Considering the specific characteristics of the express delivery signing scenario, an interactive OCR module is specifically introduced. This module allows the model to dynamically respond to complex features such as blurring, tilting, and occlusion of the watermark text during the recognition process, significantly improving the recognition accuracy and robustness in complex scenarios, and solving the problem of insufficient recognition capability of traditional OCR for special-shaped watermarks. During the training phase, the annotated dataset undergoes multi-dimensional data augmentation to cover various interference factors that may occur in actual signing scenarios. Then, the augmented dataset is used to train the basic model. Simultaneously, the CTC loss function is used to calculate the loss between the predicted sequence and the annotated sequence, strengthening the model's generalization ability to recognize watermark text in different scenarios. Finally, the trained watermark text recognition model is used to analyze the real-time uploaded delivery photos, accurately extracting the structured information from the watermark and automatically comparing it with the actual delivery information recorded by the logistics system. Through these models and steps, fake photo-taking behavior can be effectively identified.

[0024] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the photo watermark recognition method in this invention includes: S101. Obtain multiple photo samples taken by couriers when they come to sign for packages, and annotate the watermark text in the photo samples to obtain an annotated dataset; S102. Construct the GOT-OCR2 model as the basic model for watermark text recognition, and introduce an interactive OCR module into the basic model; S103. Perform data augmentation on the labeled dataset, use the data augmented labeled dataset to train the base model, and use the CTC loss function to calculate the loss between the predicted sequence and the labeled sequence to obtain the watermark text recognition model. S104. Use a watermark text recognition model to recognize the real-time uploaded signed-for photos, output the recognition information, compare the recognition information with the actual signed-for information in the logistics system, and output the comparison result.

[0026] In this embodiment, the GOT-OCR2 model is a supervised learning model. The learning process of this type of model requires paired data of "input data + corresponding correct answer". The photo samples are the "input data", but the model cannot independently determine which part is the watermark text. Therefore, the watermark text in the photo samples needs to be labeled beforehand. When collecting photo samples taken by couriers when signing for packages, it is important to cover scenes with different lighting, backgrounds, and resolutions to ensure sample diversity. At the same time, professional labeling tools that support regional coordinate labeling and attribute labeling, such as LabelStudio, are selected to label the samples.

[0027] After obtaining a labeled dataset suitable for training, this embodiment further constructs the GOT-OCR2 model as the basic framework for watermark text recognition. GOT-OCR2 is a unified, end-to-end model with 580 million parameters, consisting of a high-compression encoder and a long context decoder, capable of processing various artificial optical signals including text, formulas, tables, charts, musical scores, and geometric shapes. By introducing an interactive OCR module, which can automatically focus on the key feature regions of the watermark text in a photo, the model's recognition accuracy is enhanced, especially in complex backgrounds or blurry photos, enabling more accurate extraction of watermark content.

[0028] During the model training phase, to further improve the model's recognition ability, this embodiment employs data augmentation to artificially expand the training data without increasing the cost of collecting the original samples. This simulates the complex variations in watermarks on express delivery receipt photos caused by shooting conditions. In the data formation process, watermark text recognition is essentially a "sequence matching task." The predicted sequence output by the model (such as the recognized character sequence) and the labeled sequence (the actual watermark text) often have length differences. The CTC loss function can solve the alignment problem of text recognition and improve sequence matching accuracy.

[0029] Finally, by employing a watermark text recognition model to identify the real-time uploaded delivery photos, and comparing the identified information with the actual delivery information in the logistics system to obtain the final comparison result, the system can automatically identify watermarks in the delivery photos. This reduces reliance on manual review of photo watermarks, lowers labor costs and review workload, and improves the efficiency of express delivery delivery management. Simultaneously, accurate watermark text recognition can effectively prevent fraudulent photography, ensuring consumers receive genuine delivery information, thereby enhancing the consumer experience.

[0030] Please see Figure 2 The second embodiment of the photo watermark recognition method in this invention includes: S201. Obtain multiple photo samples taken by the courier when signing for the package, draw a bounding box for the watermark text in each photo, and record the pixel coordinates of the upper left and lower right corners of the bounding box; S202. The font characteristics of the watermark text, wherein the font characteristics include font type, font size, font weight, and color; S203. Format rules for watermark content to clarify the composition structure of each field in order to obtain the labeled dataset.

[0031] In this embodiment, to ensure the quality of the labeled data, photos need to be labeled from multiple directions and angles, such as time, location, and order number. For the watermark location, a bounding box is drawn, and the pixel coordinates of the upper left and lower right corners are recorded to accurately locate the watermark. For semi-transparent watermarks and watermarks similar in color to the background, a clear bounding box can guide the model to focus on key areas, improving the watermark detection rate in complex scenes. For watermark text, features such as font type, font size, font weight, and color are labeled. Through labeling, the model can learn the unique visual fingerprint of the watermark, for example, distinguishing between "semi-transparent light gray 12-point text" (watermark) and "opaque black 16-point text" (shipping label), thereby reducing misidentification of non-watermark text and enhancing the targeted recognition ability of watermark text. For watermark content such as time and order number, their format rules are recorded, clarifying the composition structure of each field. The labeled sample data can be divided into a training set, a validation set, and a test set in a 7:2:1 ratio for subsequent model training and evaluation.

[0032] Please see Figure 3 The third embodiment of the photo watermark recognition method in this invention includes: S301. Construct a high-compression encoder, the high-compression encoder comprising an input layer, a convolutional layer module and a first Transformer encoder module arranged sequentially, the convolutional layer module comprising multiple convolutional layers, each convolutional layer being followed by a BatchNorm and a ReLU activation function, the first Transformer encoder module being composed of a multi-head self-attention mechanism and a feedforward neural network; S302. Construct a long context decoder, the long context decoder including a second Transformer encoder module and an output layer arranged sequentially, the second Transformer encoder module including a multi-head self-attention mechanism and an encoder-decoder attention mechanism, the output layer adopting a linear mapping and a Softmax function; S303. Construct the GOT-OCR2 model based on the high-compression encoder and long-context decoder, and introduce an interactive OCR module into the GOT-OCR2 model.

[0033] In this embodiment, constructing the GOT-OCR2 model specifically includes the following steps: First, a high-compression encoder is constructed, comprising an input layer, a convolutional layer module, and a first Transformer encoder module arranged sequentially. The input layer receives preprocessed image data, uniformly sized to 3×H×W, where 3 represents the RGB channels. The data then passes through the convolutional layer module, which contains multiple convolutional operations. Each convolutional layer is followed by a BatchNorm and ReLU activation function to progressively extract local features of the image, such as edges and textures. The feature maps output by the convolutional layers are then fed into the first Transformer encoder module, which consists of a multi-head self-attention mechanism and a feedforward neural network. This module performs global feature association learning on the feature maps output by the convolutional layers, compressing the image information into a feature sequence. Then, a long context decoder is constructed. The long context decoder takes the feature sequence output by the high-compression encoder as input and combines it with position encoding to capture the position information of the sequence. Through the second Transformer decoder module (which includes a multi-head self-attention mechanism and an encoder-decoder attention mechanism), the feature sequence is modeled with long context dependency to process the sequence features of different types of optical signals. The output layer uses linear mapping and the Softmax function to convert the decoder output into a character probability distribution, thereby achieving a unified recognition output for multiple optical signals.

[0034] Next, a GOT-OCR2 model is constructed based on the high-compression encoder and the long context decoder, and an interactive OCR module is introduced into the GOT-OCR2 model. The interactive OCR module is used in conjunction with the high-compression encoder to obtain an enhanced watermark feature sequence, which is then input into the decoder for recognition.

[0035] Please see Figure 4 The fourth embodiment of the photo watermark recognition method in this invention includes: S401. Construct a GOT-OCR2 model based on a high-compression encoder and a long context decoder as the base model; S402. Construct a coordinate-guided region-level recognition submodule to generate a feature map based on a region mask; construct a color-guided region-level recognition submodule to generate a feature map based on a color mask; construct a fusion submodule to perform a weighted summation of the feature map based on the region mask and the feature map based on the color mask to obtain a watermark feature sequence. S403. Integrate the coordinate-guided region-level recognition submodule, the color-guided region-level recognition submodule, and the fusion submodule to obtain an interactive OCR module, and introduce the interactive OCR module into the GOT-OCR2 model.

[0036] In this embodiment, the interactive OCR module includes a coordinate-guided region-level recognition submodule and a coordinate-guided region-level recognition submodule. The coordinate-guided region-level recognition submodule receives the coordinates of the watermark text bounding box recorded during the annotation stage, generates a region mask, with the area inside the mask representing the watermark region and the area outside the mask representing the background region. During the model feature extraction stage, the region mask is multiplied element-wise with the feature map output by the high-compression encoder to strengthen the feature weights of the area within the mask and suppress feature interference from the background region. The color-guided region-level recognition submodule, based on the color features of the annotated watermark text, sets a color threshold range (e.g., RGB values ​​fluctuating by 10%), performs color segmentation on the input image, extracts pixel areas that meet the threshold range as candidate watermark regions, generates a color mask, and multiplies the color mask with the encoder feature map to further focus on the color feature region of the watermark text. Through these two region-level recognition submodules, coordinate-guided and color-guided results can be obtained. Then, a fusion submodule merges the coordinate-guided and color-guided results, performs a weighted summation of the feature maps generated by the two guidance mechanisms, obtains an enhanced watermark feature sequence, and then inputs it into the long context decoder for recognition. The interactive OCR module uses coordinate- or color-guided region-level recognition to automatically focus on key feature areas of watermark text in photos. Based on a high-compression encoder, it enhances the model's accuracy in recognizing watermark text, especially in complex backgrounds or blurry photos, enabling more accurate extraction of watermark content.

[0037] Please see Figure 5 The fifth embodiment of the photo watermark recognition method in this invention includes: S501. Perform data augmentation processing on the labeled dataset, the data augmentation processing including rotating, scaling, adjusting watermark text, adjusting color, adjusting image lighting parameters, adding noise and blurring the image; S502. Input the data-augmented labeled dataset into the base model to obtain the predicted character sequence; S503. The CTC loss function is used to calculate the loss between the predicted sequence and the labeled sequence, and the Adam optimizer is used to update the model parameters through the backpropagation algorithm to obtain the watermark text recognition model.

[0038] In this embodiment, the data enhancement processing can specifically employ the following operations: during rotation enhancement, the image can be rotated at a random angle from -15° to 15°; while maintaining the integrity of the watermark text, the image can be scaled at a random ratio from 0.8 to 1.2 times; the size of the watermark text can be adjusted; the image color can be randomly adjusted; the brightness, contrast, and saturation of the image can be randomly adjusted to simulate different lighting conditions; Gaussian noise can be added to the image; and the image can be blurred to simulate a blurred shooting effect.

[0039] During model training, the Xavier initialization method was used to initialize the model parameters, setting the batch size to 32 and the initial learning rate to 1e-4. Then, the enhanced training set images were input into the model, processed by the encoder and decoder to obtain the predicted character sequences. The CTC loss function was then used to calculate the loss between the predicted and labeled sequences. Whitespace characters were inserted into both sequences to handle sequences of unequal length. A dynamic programming algorithm was used to calculate the sum of probabilities of all possible alignment paths, and the negative logarithm was used as the loss value. Next, the Adam optimizer was used to update the model parameters through backpropagation, calculating the gradient of the loss with respect to each parameter and adjusting the parameters according to the gradient descent direction. Every 10 epochs, the model performance was evaluated using a validation set, and the recognition accuracy was calculated. If the validation set accuracy did not improve for three consecutive epochs, the learning rate was reduced, and the training process was repeated until a preset maximum epoch, such as 300, was reached or the model converged.

[0040] Please see Figure 6 The sixth embodiment of the photo watermark recognition method in this invention includes: S601. Obtain the real-time uploaded signed-out photos, adjust the size of the signed-out photos, and perform grayscale normalization processing; S602. Use a watermark text recognition model to recognize the real-time uploaded signed photos and output the recognition information; S603. Extract key information from the identification information and compare it with the actual receipt information in the logistics system to obtain the comparison result. The key information includes the expected receipt time, receipt location, and system record number. S604. If any field in the comparison result does not match, the system will automatically trigger an early warning signal.

[0041] In this embodiment, before using the constructed watermark text recognition model to recognize the real-time uploaded delivery photos, the delivery photos uploaded by the courier need to be preprocessed. This includes automatically adjusting the image size to the model's input size, performing grayscale normalization, and reducing the impact of lighting differences. After inputting the preprocessed image into the watermark text recognition model, features are extracted by a high-compression encoder, enhanced by an interactive OCR module, and then the recognition result is output by a long context decoder. The recognition result includes the result, time, location, tracking number, and other strings. Next, the recognition result is compared with the actual information at the field level to obtain the comparison result.

[0042] Please see Figure 7 The seventh embodiment of the photo watermark recognition method in this invention includes: S701. Extract key information from the identification information and compare it with the actual receipt information in the logistics system. The key information includes the estimated receipt time, receipt location, and system record number. S702. Check whether the identified estimated receipt time string conforms to the preset format and whether the difference between it and the actual time is within the allowable range; S703. Use a string similarity algorithm to calculate the similarity between the identified delivery location and the actual location. If the similarity is lower than the set threshold, it is determined to be inconsistent. S704. Perform a string-by-string match between the identified system record number and the system record number in the logistics system. If they do not match, they are determined to be mismatched.

[0043] In this embodiment, the comparison of key information includes the following steps: For the estimated receipt time identified by the model, check whether its time string conforms to the preset format and whether the difference between it and the actual time is within the allowable range. For example, the model-identified time "2024-10-29 16:50" conforms to the preset format of "year-month-day hour:minute", and the format verification passes; if the identification result is "2024 / 10 / 29 16:50", then the time field is determined to be inconsistent. During the difference verification, the difference between the identified time (16:50) and the actual time (16:45) is calculated to be 5 minutes, which is within the allowable range of ≤30 minutes. Therefore, the time field is ultimately determined to be "compliant".

[0044] When comparing locations, if the identified location is "No. 88, CCC Road, BB District, AA City", and the actual location in the system is "No. 88, CCC Road, BB District, AA City", then in the similarity calculation, the two only differ in the last character, and the edit distance is 1; the total number of characters is 13, and the similarity = (1-1 / 13)×100%≈92.3%, which is higher than the set threshold of 80%, then it is judged as "matching".

[0045] The tracking number is a unique identifier. Therefore, when comparing tracking numbers, it is necessary to strictly adhere to the requirement that all characters must be completely identical. There is no room for error; if they are inconsistent, the tracking number will be directly determined to be a mismatch.

[0046] The above describes the method for recognizing watermarks on signed receipt photos in embodiments of the present invention. The following describes the device for recognizing watermarks on signed receipt photos in embodiments of the present invention. Please refer to [link / reference]. Figure 8 One embodiment of the photo watermark recognition device for signature in this invention includes: The annotation module 10 is used to acquire multiple photo samples taken by couriers when they come to sign for packages, and to annotate the watermark text in the photo samples to obtain an annotated dataset. Module 20 is used to build the GOT-OCR2 model as the basic model for watermark text recognition, and to introduce an interactive OCR module into the basic model; Training module 30 is used to perform data augmentation on the labeled dataset. The base model is trained using the data augmented labeled dataset, and the loss between the predicted sequence and the labeled sequence is calculated using the CTC loss function to obtain the watermark text recognition model. The recognition module 40 is used to recognize the real-time uploaded signed photos using a watermark text recognition model, output the recognition information, compare the recognition information with the actual signed information in the logistics system, and output the comparison result.

[0047] Please see Figure 9 One embodiment of the photo watermark recognition device for signature in this invention includes: The annotation module 10 is used to acquire multiple photo samples taken by couriers when they come to sign for packages, and to annotate the watermark text in the photo samples to obtain an annotated dataset. Module 20 is used to build the GOT-OCR2 model as the basic model for watermark text recognition, and to introduce an interactive OCR module into the basic model; Training module 30 is used to perform data augmentation on the labeled dataset. The base model is trained using the data augmented labeled dataset, and the loss between the predicted sequence and the labeled sequence is calculated using the CTC loss function to obtain the watermark text recognition model. The recognition module 40 is used to recognize the real-time uploaded signed photos using a watermark text recognition model, output the recognition information, compare the recognition information with the actual signed information in the logistics system, and output the comparison result. In this embodiment, the annotation module 10 includes: Boundary annotation unit 11 is used to acquire multiple photo samples taken by couriers when they come to sign for packages, draw bounding boxes for the watermark text in each photo, and record the pixel coordinates of the upper left and lower right corners of the bounding boxes. The font annotation unit 12 is used to annotate the font features of the watermark text, including font type, font size, font weight, and color. Format annotation unit 13 is used to annotate the format rules of the watermark content to clarify the composition structure of each field in order to obtain the annotated dataset; In this embodiment, the construction module 20 includes: The first building unit 21 is used to build a high-compression encoder. The high-compression encoder includes an input layer, a convolutional layer module and a first Transformer encoder module arranged in sequence. The convolutional layer module includes multiple convolutional layers. Each convolutional layer is followed by a BatchNorm and a ReLU activation function. The first Transformer encoder module is composed of a multi-head self-attention mechanism and a feedforward neural network. The second construction unit 22 is used to construct a long context decoder. The long context decoder includes a second Transformer encoder module and an output layer arranged in sequence. The second Transformer encoder module includes a multi-head self-attention mechanism and an encoder-decoder attention mechanism. The output layer adopts a linear mapping and a Softmax function. The third building unit 23 is used to build a GOT-OCR2 model based on a high-compression encoder and a long context decoder, and to introduce an interactive OCR module into the GOT-OCR2 model; In this embodiment, the third building unit 23 includes: Model subunit 231 is used to construct a GOT-OCR2 model based on a high-compression encoder and a long context decoder as a base model; Subunit 232 is constructed to construct a coordinate-guided region-level recognition submodule to generate a feature map based on a region mask, a color-guided region-level recognition submodule to generate a feature map based on a color mask, and a fusion submodule to perform a weighted summation of the feature map based on the region mask and the feature map based on the color mask to obtain a watermark feature sequence. Integration subunit 233 is used to integrate the coordinate-guided region-level recognition submodule, the color-guided region-level recognition submodule, and the fusion submodule to obtain an interactive OCR module, and to introduce the interactive OCR module into the GOT-OCR2 model. In this embodiment, the training module 30 includes: Data augmentation unit 31 is used to perform data augmentation processing on the labeled dataset. The data augmentation processing includes rotating, scaling, adjusting watermark text, adjusting color, adjusting image lighting parameters, adding noise, and blurring the image. The first training unit 32 is used to input the data-augmented labeled dataset into the base model to obtain predicted character sequences; The second training unit 33 is used to calculate the loss between the predicted sequence and the labeled sequence using the CTC loss function, and to update the model parameters using the Adam optimizer and the backpropagation algorithm to obtain the watermark text recognition model. In this embodiment, the identification module 40 includes: The preprocessing unit 41 is used to acquire the real-time uploaded signed-out photos, adjust the size of the signed-out photos, and perform grayscale normalization processing. The recognition unit 42 is used to recognize the real-time uploaded signature photos using a watermark text recognition model and output the recognition information. The comparison unit 43 is used to extract key information from the identification information and compare it with the actual receipt information in the logistics system to obtain the comparison result. The key information includes the expected receipt time, receipt location, and system record number. The early warning unit is used to automatically trigger an early warning signal if any field in the comparison result does not match. In this embodiment, the comparison unit 43 includes: Extraction subunit 431 is used to extract key information from the identification information and compare it with the actual receipt information in the logistics system. The key information includes the expected receipt time, receipt location, and system record number. The first comparison subunit 432 is used to check whether the identified estimated receipt time string conforms to the preset format and whether the difference between it and the actual time is within the allowable range. The second comparison subunit 433 is used to calculate the similarity between the identified delivery location and the actual location using a string similarity algorithm. If the similarity is lower than the set threshold, it is determined to be inconsistent. The third comparison subunit 434 is used to perform a string-by-string match between the identified system record number and the system record number in the logistics system. If they do not match, they are judged as mismatched.

[0048] The watermark recognition device for signed delivery photos of this invention first collects a large number of photo samples taken by couriers when they sign for packages. It then performs detailed annotation on the watermark text in the samples, clarifying both the spatial location of the watermark and the text content and format features, thereby constructing a high-quality annotated dataset. Next, at the model construction level, the GOT-OCR2 model is selected as the basic framework for watermark text recognition. Simultaneously, considering the special characteristics of express delivery signing scenarios, such as watermarks often appearing blurry, tilted, or partially hidden due to shooting angle, lighting, or package obstruction, an interactive OCR module is specifically introduced. This allows the model to dynamically respond to these complex features during the recognition process, significantly improving the recognition accuracy of watermark text and effectively compensating for the weakness of traditional OCR in recognizing non-standard watermark shapes. Then, during the training phase, multi-dimensional data augmentation processing is performed on the annotated dataset to expand the scene coverage of the training data. The augmented dataset is then used to iteratively train the basic model, while the CTC loss function is used to calculate the loss between the predicted sequence and the annotated sequence, avoiding training bias caused by watermark text adhesion or discontinuity, further strengthening the model's generalization ability to recognize watermark text in diverse scenarios. Finally, by using the trained watermark text recognition model, the real-time uploaded signature photos by couriers are automatically parsed to accurately extract structured information such as time, location, and device number contained in the watermark; then, this information is compared with the actual signature information recorded by the logistics system at the field level.

[0049] The above describes the receipt photo watermark recognition device in the embodiments of the present invention from the perspective of modular functional entities. The following describes the receipt photo watermark recognition device in the embodiments of the present invention from the perspective of hardware processing.

[0050] Figure 10 This is a schematic diagram of a receipt photo watermark recognition device 900 provided in an embodiment of the present invention. The receipt photo watermark recognition device 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the receipt photo watermark recognition device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute a series of instruction operations in the storage media 930 on the receipt photo watermark recognition device 900 to implement the steps of the receipt photo watermark recognition method provided in the above-described method embodiments.

[0051] The signature photo watermark recognition device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 10 The illustrated structure of the signed photo watermark recognition device does not constitute a limitation on the signed photo watermark recognition device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0052] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the signature photo watermark recognition method.

[0053] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device or apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. A method for recognizing watermarks on signed receipt photos, characterized in that, Includes the following steps: We obtained multiple photo samples taken by couriers when they signed for packages at the door, and annotated the watermark text in the photo samples to obtain an annotated dataset. We constructed the GOT-OCR2 model as the basic model for watermark text recognition, and introduced an interactive OCR module into the basic model. The labeled dataset is augmented, and the base model is trained using the augmented labeled dataset. The CTC loss function is used to calculate the loss between the predicted sequence and the labeled sequence to obtain the watermark text recognition model. A watermark text recognition model is used to identify the real-time uploaded signed-for photos, output the recognition information, compare the recognition information with the actual signed-for information in the logistics system, and output the comparison result.

2. The method for recognizing watermarks on signed receipt photos according to claim 1, characterized in that, The process of obtaining multiple photo samples taken by couriers when they sign for packages at the door, and annotating the watermark text in the photo samples to obtain an annotated dataset, specifically includes: Obtain multiple photo samples taken by couriers when they come to sign for packages, draw bounding boxes for the watermark text in each photo, and record the pixel coordinates of the upper left and lower right corners of the bounding boxes; The font features of the watermark text include font type, font size, font weight, and color; The format rules for watermark content are defined to clarify the composition structure of each field in order to obtain the labeled dataset.

3. The method for recognizing watermarks on signed receipt photos according to claim 1, characterized in that, The construction of the GOT-OCR2 model serves as the foundational model for watermark text recognition, and an interactive OCR module is introduced into the foundational model, specifically including: A high-compression encoder is constructed, which includes an input layer, a convolutional layer module and a first Transformer encoder module arranged sequentially. The convolutional layer module includes multiple convolutional layers, each of which is followed by a BatchNorm and a ReLU activation function. The first Transformer encoder module is composed of a multi-head self-attention mechanism and a feedforward neural network. A long context decoder is constructed, which includes a second Transformer encoder module and an output layer arranged sequentially. The second Transformer encoder module includes a multi-head self-attention mechanism and an encoder-decoder attention mechanism. The output layer adopts a linear mapping and a Softmax function. The GOT-OCR2 model is constructed based on a high-compression encoder and a long context decoder, and an interactive OCR module is introduced into the GOT-OCR2 model.

4. The method for recognizing watermarks on signed receipt photos according to claim 3, characterized in that, The construction of the GOT-OCR2 model based on the high-compression encoder and long context decoder, and the introduction of an interactive OCR module into the GOT-OCR2 model, specifically includes: A GOT-OCR2 model was constructed based on a high-compression encoder and a long context decoder as the base model; A coordinate-guided region-level recognition submodule is constructed to generate a feature map based on a region mask; a color-guided region-level recognition submodule is constructed to generate a feature map based on a color mask; and a fusion submodule is constructed to perform a weighted summation of the feature maps based on the region mask and the feature maps based on the color mask to obtain a watermark feature sequence. The coordinate-guided region-level recognition submodule, the color-guided region-level recognition submodule, and the fusion submodule are integrated to obtain an interactive OCR module, which is then introduced into the GOT-OCR2 model.

5. The method for recognizing watermarks on signed receipt photos according to claim 1, characterized in that, The process of data augmentation of the labeled dataset, training the base model using the augmented dataset, and calculating the loss between the predicted sequence and the labeled sequence using the CTC loss function to obtain the watermark text recognition model specifically includes: Data augmentation processing is performed on the labeled dataset, including image rotation, scaling, watermark text adjustment, color adjustment, image lighting parameter adjustment, noise addition, and image blurring. The data-augmented labeled dataset is input into the base model to obtain predicted character sequences; The CTC loss function is used to calculate the loss between the predicted sequence and the labeled sequence, and the Adam optimizer is used to update the model parameters through the backpropagation algorithm to obtain the watermark text recognition model.

6. The method for recognizing watermarks on signed receipt photos according to claim 1, characterized in that, The method employs a watermark text recognition model to identify the real-time uploaded signed-for photos, outputs the identified information, compares the identified information with the actual signed-for information in the logistics system, and outputs the comparison result. Specifically, this includes: Get the real-time uploaded signed-out photos, adjust the size of the signed-out photos, and perform grayscale normalization processing; A watermark text recognition model is used to identify the real-time uploaded signed photos and output the recognition information; Key information extracted from the identification information is compared with the actual receipt information in the logistics system to obtain the comparison results. The key information includes the estimated receipt time, receipt location, and system record number. If any field in the comparison results does not match, the system will automatically trigger an early warning signal.

7. The method for recognizing watermarks on signed receipt photos according to claim 6, characterized in that, The key information extracted from the identification information is compared with the actual receipt information in the logistics system to obtain the comparison result. The key information includes the estimated receipt time, receipt location, and system record number, specifically including: Key information extracted from the identification information is compared with the actual receipt information in the logistics system. The key information includes the estimated receipt time, receipt location, and system record number. Check whether the identified estimated delivery time string conforms to the preset format and whether the difference between it and the actual time is within the allowable range; The similarity between the identified delivery location and the actual location is calculated using a string similarity algorithm. If the similarity is lower than the set threshold, it is determined to be inconsistent. The system record number is matched exactly with the system record number in the logistics system. If they do not match, they are considered mismatched.

8. A device for recognizing watermarks on signed receipts, characterized in that, include: The annotation module is used to acquire multiple photo samples taken by couriers when they come to sign for packages, and to annotate the watermark text in the photo samples to obtain an annotated dataset. The building module is used to build the GOT-OCR2 model as the basic model for watermark text recognition, and introduces an interactive OCR module into the basic model; The training module is used to perform data augmentation on the labeled dataset. The base model is trained using the data augmented labeled dataset, and the CTC loss function is used to calculate the loss between the predicted sequence and the labeled sequence to obtain the watermark text recognition model. The recognition module is used to identify the real-time uploaded signed-for photos using a watermark text recognition model, output the recognition information, compare the recognition information with the actual signed-for information in the logistics system, and output the comparison result.

9. A device for recognizing watermarks on signed receipts, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform the steps of the signature photo watermark recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the various steps of the signature photo watermark recognition method as described in any one of claims 1-7.

Images