Duplicate checking method, device and equipment for express sheet

Abstract

The invention discloses an express waybill duplicate checking method, device and equipment, and the method is applied to the technical field of logistics, and the method comprises the steps: storing historical key text data, historical express attribute data and historical waybill graphic features corresponding to a historical waybill image in a preset database; obtaining a to-be-detected express sheet image of a to-be-detected express parcel, inputting the to-be-detected express sheet image into the express sheet area detection model for target detection, and identifying key information areas in the to-be-detected express sheet image and an information category corresponding to each key information area; and determining whether the to-be-detected express sheet image is a repeated express sheet according to the key text data corresponding to the to-be-detected express sheet image, the express delivery attribute data, the express sheet graphic features and a preset database. According to the invention, the repeated express sheets can be automatically, rapidly and accurately identified.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and equipment for checking the duplicate of an express delivery bill. Background Art

[0002] With the rapid development of e-commerce, express delivery business has seen explosive growth. In the express delivery process, the delivery bill, as a key carrier of sender and recipient information, is crucial for accuracy and uniqueness. However, due to human error, system failures, and other factors, duplicate delivery bills often occur. If these duplicate delivery bills are not promptly detected and addressed, they can lead to numerous issues, including package sorting confusion, delivery delays, and logistics tracking errors. These issues significantly impact express delivery service quality and operational efficiency, increase operating costs, and even pose the risk of customer information leakage. Currently, the express delivery industry lacks efficient and accurate means to detect delivery bill duplication, relying primarily on manual visual inspection, which is not only inefficient but also prone to errors, making it difficult to meet the growing demand for express delivery services. Summary of the Invention

[0003] The present invention provides a method, device and equipment for checking duplicate express delivery bills, which can automatically, quickly and accurately identify duplicate delivery bills.

[0004] In one aspect, the present invention provides a method for checking duplicate delivery of an express delivery bill, the method comprising: Storing historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database; Obtain an image of the express parcel to be inspected, input the image into a face order region detection model to perform target detection, and identify key information regions in the face order image and the information category corresponding to each key information region; Acquire a first key information area whose information category is key text information, and extract key text data in the first key information area; Acquire a second key information area whose information category is barcode information, and extract express attribute data in the second key information area; Acquire a third key information region of the waybill image whose information category is information category, and extract the waybill graphic features in the third key information region; Determine whether the waybill image to be detected is a duplicate waybill based on the key text data, express attribute data, waybill graphic features and the preset database corresponding to the waybill image to be detected.

[0005] Optionally, before storing the historical key text data, historical express attribute data, and historical waybill graphic features corresponding to the historical waybill images in the preset database, the method further includes: Obtaining historical express delivery image of historical express parcels, and processing the historical delivery image to obtain a historical binary image; Using OCR technology to identify historical key text information in the historical binary image, and converting the historical key text information into historical key text data; Using a barcode recognition algorithm to extract historical barcode information from the historical binary image, and parsing the historical barcode information to obtain historical express attribute data of the historical express package; The historical face single graphic features are extracted from the historical binary image by using an image morphology algorithm.

[0006] Optionally, the acquiring of historical express parcel waybill images and processing the historical waybill images to obtain historical binary images includes: Get historical express parcel waybill images; Grayscale processing is performed on the historical face image to obtain a historical grayscale image; Using a Gaussian filtering algorithm to remove noise from the historical grayscale image to obtain a historical denoised image; An adaptive threshold segmentation method is adopted to automatically determine a threshold according to the local grayscale characteristics of the historical denoised image, so as to separate the text and barcode in the historical denoised image from the background and obtain the historical binary image.

[0007] Optionally, determining whether the waybill image to be detected is a duplicate waybill based on key text data, express attribute data, waybill graphic features corresponding to the waybill image to be detected, and the preset database includes: Calculating a first similarity between each historical key text data in the preset database and the key text data; Determine the historical express delivery image corresponding to the historical key text data whose first similarity is greater than the first threshold as the screening express delivery image, and obtain the historical express delivery attribute data and the historical express delivery graphic features corresponding to the screening express delivery image as the screening express delivery attribute data and the screening express delivery graphic features; Calculating a second similarity between the express attribute data and the filtered express attribute data and a third similarity between the waybill graphic feature and the filtered waybill graphic feature; If the second similarity is greater than a second threshold and the third similarity is greater than a third threshold, it is determined that the face order image to be detected is a duplicate face order.

[0008] Optionally, after determining that the face order image to be detected is a duplicate face order, the method further includes: Obtaining the image of the express package to be inspected, key text data, express attribute data, express label graphic features, express label number, and express label comparison results, generating duplicate express label information and triggering an alarm mechanism; the express label comparison results include the first similarity, the second similarity, and the third similarity; The duplicate bill of lading information is sent to the terminal of the auditing user so that the auditing user can verify the duplicate bill of lading information and take corrective measures to correct the duplicate bill of lading.

[0009] Optionally, the training method of the face sheet area detection model includes: Obtain a sample face sheet image of a sample express package; the sample face sheet image is annotated with a sample key information area label and a sample information category label corresponding to the sample key information area; the sample key information area includes a sample key text information area, a sample barcode information area, and a sample face sheet image area; Input the sample face single image into the YOLOv8 model for region prediction and region information category prediction to obtain the sample key information region result and the sample information category result; determining target loss data based on a difference between the sample key and information region result and the sample key information region label, and a difference between the sample information category result and the sample information category label; The model parameters of the YOLOv8 model are adjusted based on the target loss data until the training end condition is met, and the YOLOv8 model at the end of training is determined as the face single area detection model.

[0010] Optionally, the YOLOv8 model includes a backbone network, a neck network, and a head network. Inputting the sample face single image into the YOLOv8 model for region prediction and region information category prediction to obtain a sample key information region result and a sample information category result includes: Extracting image features and text features from the sample face image based on the backbone network; Based on the neck network, the image features and the text features are fused and enhanced to obtain sample area features; Based on the head network, regional prediction and regional information category prediction are performed on the sample region features to obtain sample key information region results and sample information category results.

[0011] Another aspect provides a device for checking duplicated content of an express delivery bill, the device comprising: A historical waybill feature storage module is used to store historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database; A key information area recognition module is used to obtain an image of the express parcel to be inspected, input the image into the parcel area detection model to perform target detection, and identify the key information areas in the parcel image and the information category corresponding to each key information area; A key text data extraction module, configured to obtain a first key information region whose information category is key text information, and extract key text data from the first key information region; an express attribute data extraction module, configured to obtain a second key information area whose information category is barcode information, and extract express attribute data from the second key information area; a waybill graphic feature extraction module, configured to obtain a third key information region of the waybill image whose information category is the waybill image, and extract the waybill graphic features in the third key information region; A duplicate bill determination module is used to determine whether the bill image to be detected is a duplicate bill based on the key text data, express attribute data, bill graphic features and the preset database corresponding to the bill image to be detected.

[0012] On the other hand, an electronic device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the express delivery bill duplicate checking method as described above.

[0013] On the other hand, a computer storage medium is provided, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the express delivery bill duplicate checking method as described above.

[0014] Another aspect provides a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the above-described method for checking duplicated courier delivery notes.

[0015] The method, device, and equipment for checking duplicate delivery bills provided by the present invention have the following technical effects: The present invention stores historical key text data, historical express attribute data, and historical express graphic features corresponding to historical express images in a preset database; obtains an express image of a to-be-detected express package, and inputs the to-be-detected express image into an express region detection model for target detection, identifying key information regions in the to-be-detected express image and the information categories corresponding to each key information region; obtains a first key information region whose information category is key text information and extracts key text data in the first key information region; obtains a second key information region whose information category is barcode information and extracts express attribute data in the second key information region; obtains a third key information region whose information category is the express image and extracts express graphic features in the third key information region; and determines whether the to-be-detected express image is a duplicate express image based on the key text data, express attribute data, express graphic features, and the preset database corresponding to the to-be-detected express image. The present invention can automatically, quickly, and accurately identify duplicate express images, avoid misdelivery or missed delivery, and ensure accurate delivery. Automated detection reduces manual review time, improves efficiency, reduces error costs, and avoids express loss, damage compensation, and customer complaint handling costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions and advantages of the embodiments of this specification or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a schematic diagram of a duplicate checking system for express delivery bills provided in an embodiment of this specification; Figure 2 This is a flow chart of a method for checking duplicated express delivery bills provided in an embodiment of this specification; Figure 3 This is a flow chart of a method for extracting graphic features of historical shipping documents provided in an embodiment of this specification; Figure 4 This is a flow chart of a method for obtaining a historical express parcel's historical waybill image and processing the historical waybill image to obtain a historical binary image, provided in an embodiment of this specification; Figure 5 1 is a flow chart of a method for determining whether the face order image to be detected is a duplicate face order, provided in an embodiment of this specification; Figure 6 This is a schematic diagram of the structure of a duplicate checking device for an express delivery bill provided in an embodiment of this specification; Figure 7 This is a structural diagram of a server provided in an embodiment of this specification. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] See also Figure 1 , Figure 1 This is a schematic diagram of a duplicate checking system for an express delivery bill provided in an embodiment of this specification. Figure 1 As shown, the express delivery bill duplicate checking system may include at least a server 01 and a client 02.

[0021] Specifically, in the embodiments of this specification, the server 01 may include an independently running server, or a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, a memory, and the like. Specifically, the server 01 can be used to train a face order area detection model and determine whether the face order image to be detected is a duplicate face order based on the key text data, express attribute data, face order graphic features, and the preset database corresponding to the face order image to be detected.

[0022] Specifically, in the embodiments of this specification, the client 02 may include a physical device such as a smartphone, desktop computer, tablet computer, laptop computer, digital assistant, smart wearable device, smart speaker, in-vehicle terminal, smart TV, etc. It may also include software running on the physical device, such as a web page provided by a service provider to a user, or an application provided by the service provider to a user. Specifically, the client 02 may be used to display information about duplicate shipping documents.

[0023] The following describes a method for checking duplicate delivery bills of the present invention. Figure 2 It is a flow chart of a method for checking duplicates of express delivery labels provided in the embodiment of this specification. This specification provides the method operation steps as described in the embodiment or flow chart, but it may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, the method may include: S201: Storing historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database. In the embodiments of this specification, historical key text data may be the recipient's name, address, telephone number, and sender-related information in the historical waybill image; historical express attribute data may include information such as the express number and logistics code; historical waybill graphic features may include features such as the shape, size, position and form of specific identification patterns of the waybill.

[0024] S203: Obtain the face order image to be detected of the express package to be detected, and input the face order image to be detected into the face order area detection model for target detection, and identify the key information areas in the face order image to be detected and the information category corresponding to each key information area.

[0025] In the embodiments of this specification, the key information area may include a first key information area corresponding to key text information, a second key information area corresponding to barcode information, and a third key information area corresponding to the face sheet image. Information categories include key text information, barcode information, and face sheet image.

[0026] When a face order image to be inspected is fed into the face order region detection model, the model rapidly scans the image and, based on the learned feature patterns, identifies key information areas within the face order. It then outputs a category label and the corresponding bounding box coordinates for each area. Furthermore, the model extracts text information from these identified key information areas. Combining OCR and barcode recognition techniques, the model compares this feature information with historical face order feature information stored in a database. This comparison utilizes various similarity calculation methods, such as edit distance and cosine similarity for textual information to measure the similarity of character sequences; shape descriptor-based similarity calculations and structural similarity indices for graphical features to assess the similarity of face order shape, pattern layout, and other aspects; and direct, precise matching for barcode features. The model combines the similarity results across all dimensions. If the overall similarity exceeds a pre-set threshold, the face order is deemed a duplicate.

[0027] S205: Acquire a first key information region whose information category is key text information, and extract key text data in the first key information region.

[0028] S207: Acquire a second key information area whose information category is barcode information, and extract express attribute data in the second key information area.

[0029] S209: Acquire a third key information region of the waybill image whose information category is the waybill image, and extract the waybill graphic features in the third key information region.

[0030] S2011: Determine whether the waybill image to be detected is a duplicate waybill based on the key text data, express attribute data, waybill graphic features and the preset database corresponding to the waybill image to be detected.

[0031] In an embodiment of the present specification, the key text data, express attribute data, and graphic features of the waybill corresponding to the waybill image to be detected can be compared with the historical waybill images in the preset database. If there is a historical waybill image that is consistent with the comparison, the waybill image to be detected is determined to be a duplicate waybill; if there is no historical waybill image that is consistent with the comparison, the waybill image to be detected is determined to be a non-duplicate waybill.

[0032] In the embodiment of this specification, before storing the historical key text data, historical express attribute data and historical waybill graphic features corresponding to the historical waybill image in the preset database, such as Figure 3 As shown, the method further includes: S301: Acquire historical express delivery bill images of historical express packages, and process the historical delivery bill images to obtain historical binary images; S303: using OCR technology to identify historical key text information in the historical binary image, and converting the historical key text information into historical key text data; S305: extracting historical barcode information from the historical binary image using a barcode recognition algorithm, and parsing the historical barcode information to obtain historical express attribute data of the historical express package; S307: Extracting historical face pattern features from the historical binary image using an image morphology algorithm.

[0033] In the embodiments of this specification, feature extraction is performed on the pre-processed binary image. On the one hand, OCR technology is used to identify key text information on the waybill, including the recipient's name, address, phone number, and sender-related information, and this text information is converted into text data storage. On the other hand, a barcode recognition algorithm is used to extract the barcode information on the waybill, parsing the data contained in the barcode, such as the express delivery number and logistics code. At the same time, an image morphology algorithm is used to extract the graphic features of the waybill, such as the shape, size, position and form of specific identification patterns, etc. These multi-dimensional features will serve as an important basis for determining whether the waybill is duplicated.

[0034] OCR technology performs text recognition on pre-processed binary images, loading a deep learning model trained on a large number of express delivery label samples. This model is based on a convolutional neural network (CNN) and recurrent neural network (RNN) architecture. The CNN component automatically extracts text features from the image, while the RNN component performs sequence modeling on the extracted features to identify the character sequence. During the recognition process, the model scans and recognizes the text area in the image line by line and character by character, achieving high recognition accuracy for text on the express delivery label in various fonts, sizes, handwriting, and print quality. Key text information, including the recipient's name, address, phone number, and sender information, is converted into text data. A barcode recognition algorithm is used to extract the barcode information on the express delivery label, using the barcode's geometric and grayscale features to locate the barcode within the binary image. Once located, decoding is performed according to the encoding rules of the different barcode types.

[0035] With the help of image morphological algorithms, the graphic features of the face sheet are extracted. Erosion and dilation operations are used to refine or thicken the lines and contours in the image, determine the shape boundary of the face sheet, and calculate the geometric parameters such as the perimeter, area, and aspect ratio of the face sheet. Through template matching or feature point detection methods, the position and shape of the specific identification pattern on the face sheet are identified, and its relative coordinates, rotation angle, scaling ratio and other information are recorded.

[0036] In the embodiments of this specification, Figure 4As shown, the method of obtaining a historical express parcel's historical waybill image and processing the historical waybill image to obtain a historical binary image includes: S3011: Obtain historical express parcel waybill images; S3013: grayscale processing is performed on the historical face image to obtain a historical grayscale image; S3015: Using a Gaussian filtering algorithm to remove noise from the historical grayscale image to obtain a historical denoised image; S3017: Adopting an adaptive threshold segmentation method, automatically determining a threshold according to the local grayscale characteristics of the historical denoised image, separating the text and barcode in the historical denoised image from the background, and obtaining the historical binary image.

[0037] In an embodiment of the present specification, a collected face order image is received, and the image is first grayscaled to convert the color image into a grayscale image, thereby reducing the amount of data while retaining the key information of the image. Then, a Gaussian filtering algorithm is used to remove noise from the image. During the filtering process, the convolution kernel is slid pixel by pixel on the image, and the new value of each pixel is equal to the weighted sum of the convolution kernel and the pixel value of the corresponding image area, such as noise caused by vibration of the conveyor belt and slight fluctuations in light, so as to make the image smoother. Next, an adaptive threshold segmentation method is used to automatically determine the threshold according to the local grayscale characteristics of the image, and the text, barcode and other information in the face order image are separated from the background to obtain a binary image.

[0038] In the embodiments of this specification, Figure 5 As shown, the method of determining whether the face order image to be detected is a duplicate face order based on the key text data, express attribute data, face order graphic features and the preset database corresponding to the face order image to be detected includes: S20111: Calculating a first similarity between each historical key text data in the preset database and the key text data; S20113: Determine the historical express delivery image corresponding to the historical key text data having the first similarity greater than a first threshold as a screening express delivery image, and obtain the historical express delivery attribute data and historical express delivery graphic features corresponding to the screening express delivery image as the screening express delivery attribute data and the screening express delivery graphic features; S20115: Calculating a second similarity between the express attribute data and the filtered express attribute data and a third similarity between the waybill graphic feature and the filtered waybill graphic feature; S20117: If the second similarity is greater than a second threshold and the third similarity is greater than a third threshold, determine that the face order image to be detected is a duplicate face order.

[0039] In an embodiment of this specification, when a pre-processed face order image is input into the YOLOv8 model, the model quickly scans the image and identifies key information areas in the face order based on the feature patterns learned during training. The model outputs the category label and corresponding bounding box coordinates for each area. For the identified key information areas, the model further extracts the text information therein. Combining OCR technology and barcode recognition technology, the feature information of the current face order is compared with the feature information of historical face orders previously stored in the database. The comparison process uses a variety of similarity calculation methods, such as the edit distance algorithm and cosine similarity algorithm to measure the similarity of character sequences for text information; the shape descriptor-based similarity calculation method and the structural similarity index are used for graphic features to evaluate the similarity of the face order shape, pattern layout, etc.; and the barcode features are directly matched accurately. The similarity results of each dimension are combined. When the overall similarity exceeds a pre-set threshold, the current face order is determined to be a duplicate face order.

[0040] The first similarity can be calculated by measuring the similarity of character sequences using algorithms such as edit distance and cosine similarity. A second similarity can be calculated between the express attribute data in the barcode information and the selected express attribute data. A third similarity can be calculated by evaluating the similarity of the shape, pattern layout, and other aspects of the delivery labels using similarity calculation methods based on shape descriptors and structural similarity indices. The first, second, and third thresholds can be the same or different and can be set based on actual circumstances.

[0041] In some embodiments, the similarity between the waybill image to be detected and each historical waybill image can also be calculated based on the key text data, express attribute data, waybill graphic features corresponding to the waybill image to be detected and the preset database; specifically, for any historical waybill image, the first similarity between the key text data and the historical key text data of the historical waybill image can be calculated, the second similarity between the express attribute data and the historical express attribute data of the historical waybill image can be calculated, and the third similarity between the waybill graphic features and the historical waybill graphic features of the historical waybill image can be calculated; then the average value of the first similarity, the second similarity, and the third similarity is calculated to obtain the similarity between the waybill image to be detected and the historical waybill image; then the difference between the similarity and the preset threshold is compared. When the similarity is greater than the preset threshold, it is determined that the waybill corresponding to the waybill image to be detected is a duplicate waybill.

[0042] In the embodiment of this specification, the training method of the face order area detection model includes: Obtain a sample face sheet image of a sample express package; the sample face sheet image is annotated with a sample key information area label and a sample information category label corresponding to the sample key information area; the sample key information area includes a sample key text information area, a sample barcode information area, and a sample face sheet image area; Input the sample face single image into the YOLOv8 model for region prediction and region information category prediction to obtain the sample key information region result and the sample information category result; determining target loss data based on a difference between the sample key and information region result and the sample key information region label, and a difference between the sample information category result and the sample information category label; The model parameters of the YOLOv8 model are adjusted based on the target loss data until the training end condition is met, and the YOLOv8 model at the end of training is determined as the face single area detection model.

[0043] In an embodiment of the present specification, a large number of image samples containing various types of express delivery bills are collected in advance, covering bills from different express companies, different formats, different printing qualities, and photographed in different environments. These samples are annotated, and key information areas in the bills are marked, such as the recipient information area, the sender information area, the barcode area, the express company logo area, etc. Each area is labeled with a specific category label. The YOLOv8 model is trained using these annotated samples, and then the YOLOv8 model is trained based on the difference between the sample key and information area results and the sample key information area labels, as well as the difference between the sample information category results and the sample information category labels, to obtain the bill area detection model.

[0044] In the embodiment of this specification, the YOLOv8 model includes a backbone network, a neck network, and a head network. The sample face single image is input into the YOLOv8 model for region prediction and region information category prediction to obtain the sample key information region result and the sample information category result, including: Extracting image features and text features from the sample face image based on the backbone network; Based on the neck network, the image features and the text features are fused and enhanced to obtain sample area features; Based on the head network, regional prediction and regional information category prediction are performed on the sample region features to obtain sample key information region results and sample information category results.

[0045] In the embodiments of this specification, the YOLOv8 model consists of a backbone network, a neck network, and a head network. The backbone network is responsible for extracting deep features of the image, the neck network fuses and enhances the features, and the head network classifies and positions the target based on the features. Through multiple iterative training, the weight parameters of the model are adjusted to enable it to accurately identify and locate various key information areas in the express delivery label, achieving high detection accuracy and recall rate.

[0046] In the embodiment of this specification, after determining that the face order image to be detected is a duplicate face order, the method further includes: Obtaining the image of the express package to be inspected, key text data, express attribute data, express label graphic features, express label number, and express label comparison results, generating duplicate express label information and triggering an alarm mechanism; the express label comparison results include the first similarity, the second similarity, and the third similarity; The duplicate bill of lading information is sent to the terminal of the auditing user so that the auditing user can verify the duplicate bill of lading information and take corrective measures to correct the duplicate bill of lading.

[0047] In the embodiment of this specification, when it is determined that the current waybill is a duplicate waybill, an alarm mechanism is immediately triggered.

[0048] Once the alarm is triggered, the buzzer will emit a continuous sharp alarm sound, and the warning light will flash red at a specific frequency. Through wireless communication technology, the detailed information of the duplicate waybill, including the waybill image, identified key information, comparison results with historical waybills, express delivery number, etc., will be pushed to the terminal device carried by the staff. The staff can view the detailed information at any time for rapid verification and processing, and take corresponding corrective measures, such as sorting out the duplicate packages, manually checking the sender and recipient information, contacting relevant personnel to confirm the situation, and re-planning the logistics route of the package, etc., to avoid problems such as wrong delivery and delay of packages caused by duplicate waybills, and ensure the efficient and accurate operation of express logistics.

[0049] It can be seen from the technical solutions provided in the above embodiments of this specification that the embodiments of this specification store historical key text data, historical express attribute data and historical express graphic features corresponding to historical waybill images in a preset database; obtain the waybill image to be detected of the express package to be detected, and input the waybill image to be detected into the waybill area detection model for target detection, identify the key information areas in the waybill image to be detected and the information category corresponding to each key information area; obtain a first key information area whose information category is key text information, and extract the key text data in the first key information area; obtain a second key information area whose information category is barcode information, and extract the express attribute data in the second key information area; obtain a third key information area whose information category is the waybill image, and extract the waybill graphic features in the third key information area; determine whether the waybill image to be detected is a duplicate waybill based on the key text data, express attribute data, waybill graphic features corresponding to the waybill image to be detected and the preset database. The present invention can automatically, quickly and accurately identify duplicate waybills, avoid wrong or missed delivery of parcels, and ensure accurate delivery. Automated detection reduces manual review time, improves efficiency, reduces error costs, and avoids the cost of parcel loss, damage compensation, and customer complaint handling.

[0050] The embodiment of this specification also provides a device for checking the duplicate of an express delivery bill, such as Figure 6 As shown, the device includes: The historical waybill feature storage module 610 is used to store historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database; The key information area identification module 620 is used to obtain a face order image of the express package to be inspected, input the face order image to be inspected into the face order area detection model for target detection, and identify the key information areas in the face order image to be inspected and the information category corresponding to each key information area; A key text data extraction module 630 is configured to obtain a first key information region whose information category is key text information and extract key text data from the first key information region; The express attribute data extraction module 640 is used to obtain a second key information area whose information category is barcode information and extract the express attribute data in the second key information area; The face bill graphic feature extraction module 650 is used to obtain a third key information area of ​​the face bill image, and extract the face bill graphic features in the third key information area; The duplicate bill determination module 660 is used to determine whether the bill image to be detected is a duplicate bill based on the key text data, express attribute data, bill graphic features and the preset database corresponding to the bill image to be detected.

[0051] In some embodiments, the apparatus further comprises: A historical binary image acquisition module is used to acquire historical express parcel waybill images and process the historical waybill images to obtain historical binary images; A historical key text conversion module is used to identify historical key text information in the historical binary image using OCR technology and convert the historical key text information into historical key text data; A historical express attribute data parsing module, configured to extract historical barcode information from the historical binary image using a barcode recognition algorithm, and parse the historical barcode information to obtain historical express attribute data of the historical express package; The historical face order graphic feature extraction module is used to extract the historical face order graphic features from the historical binary image through an image morphology algorithm.

[0052] In some embodiments, the historical binary image acquisition module includes: A historical waybill image acquisition unit, used to acquire historical waybill images of historical express parcels; A historical grayscale image acquisition unit, configured to perform grayscale processing on the historical face image to obtain a historical grayscale image; a denoising unit, configured to remove noise from the historical grayscale image using a Gaussian filtering algorithm to obtain a historical denoised image; The historical binary image determination unit is used to adopt an adaptive threshold segmentation method to automatically determine a threshold according to the local grayscale characteristics of the historical denoised image, separate the text and barcode in the historical denoised image from the background, and obtain the historical binary image.

[0053] In some embodiments, the duplicate order determination module includes: A first similarity calculation unit, configured to calculate a first similarity between each historical key text data in the preset database and the key text data; a data screening unit, configured to determine a historical express delivery image corresponding to the historical key text data having the first similarity greater than a first threshold as a screening express delivery image, and obtain historical express delivery attribute data and historical express delivery graphic features corresponding to the screening express delivery image as the screening express delivery attribute data and the screening express delivery graphic features; A third similarity calculation unit is used to calculate a second similarity between the express attribute data and the filtered express attribute data and a third similarity between the waybill graphic feature and the filtered waybill graphic feature; A duplicate face order determination unit is configured to determine that the face order image to be detected is a duplicate face order if the second similarity is greater than a second threshold and the third similarity is greater than a third threshold.

[0054] In some embodiments, the apparatus further comprises: an alarm triggering module, configured to obtain the image of the express package to be inspected, key text data, express attribute data, express label graphic features, express label number, and express label comparison results, generate duplicate express label information, and trigger an alarm mechanism; the express package comparison results include the first similarity, the second similarity, and the third similarity; The information sending module is used to send the duplicate bill information to the terminal of the reviewing user, so that the reviewing user can verify the duplicate bill information and take corrective measures to correct the duplicate bill.

[0055] In some embodiments, the apparatus further comprises: A sample image acquisition module is used to acquire a sample face sheet image of a sample express package; the sample face sheet image is annotated with a sample key information area label and a sample information category label corresponding to the sample key information area; the sample key information area includes a sample key text information area, a sample barcode information area, and a sample face sheet image area; A sample information prediction module is used to input the sample face single image into the YOLOv8 model for region prediction and region information category prediction to obtain the sample key information region result and the sample information category result; a target loss determination module, configured to determine target loss data based on a difference between the sample key and information region result and the sample key information region label, and a difference between the sample information category result and the sample information category label; A model training module is used to adjust the model parameters of the YOLOv8 model based on the target loss data until the training end condition is met, and determine the YOLOv8 model at the end of training as the face single area detection model.

[0056] In some embodiments, the YOLOv8 model includes a backbone network, a neck network, and a head network, and the sample information prediction module includes: A feature extraction unit, configured to extract image features and text features from the sample face image based on the backbone network; a feature fusion unit, configured to fuse and enhance the image features and the text features based on the neck network to obtain sample region features; The prediction unit is used to perform region prediction and region information category prediction on the sample region features based on the head network to obtain a sample key information region result and a sample information category result.

[0057] The device and method embodiments in the device embodiments are based on the same inventive concept.

[0058] An embodiment of this specification provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the method for checking the duplicate of an express delivery bill as provided in the above method embodiment.

[0059] An embodiment of the present invention also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a method for checking the duplicate of an express delivery bill in a method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the method for checking the duplicate of an express delivery bill provided in the above method embodiment.

[0060] Embodiments of the present invention further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the method for checking duplicate delivery documents provided in the above method embodiment.

[0061] Optionally, in the embodiments of this specification, the storage medium may be located in at least one of the multiple network servers in the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0062] The memory described in the embodiments of this specification can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0063] The embodiment of the express delivery bill duplicate checking method provided in the embodiment of this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 7 This is a hardware structure diagram of a server for a method of checking duplicated express delivery bills provided in an embodiment of this specification. Figure 7 As shown, the server 700 may vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 710 (CPUs 710 may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), memory 730 for storing data, and one or more storage media 720 (e.g., one or more mass storage devices) for storing applications 723 or data 722. The memory 730 and storage media 720 may be either transient or persistent storage. The program stored in the storage medium 720 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the CPU 710 may be configured to communicate with the storage medium 720 to execute the series of instruction operations in the storage medium 720 on the server 700. The server 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input and output interfaces 740, and / or one or more operating systems 721, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0064] The input / output interface 740 can be used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the server 700. In one embodiment, the input / output interface 740 may include a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the input / output interface 740 may be a radio frequency (RF) module for wireless communication with the Internet.

[0065] It can be understood by those skilled in the art that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components than shown, or with Figure 7 Different configurations shown.

[0066] It can be seen from the embodiments of the express delivery waybill checking method, device, electronic device or storage medium provided by the above-mentioned present invention that the present invention stores historical key text data, historical express attribute data and historical waybill graphic features corresponding to historical waybill images in a preset database; obtains the waybill image to be detected of the express package to be detected, and inputs the waybill image to be detected into the waybill area detection model for target detection, identifies the key information areas in the waybill image to be detected and the information category corresponding to each key information area; obtains a first key information area whose information category is key text information, and extracts the key text data in the first key information area; obtains a second key information area whose information category is barcode information, and extracts the express attribute data in the second key information area; obtains a third key information area whose information category is the waybill image, and extracts the waybill graphic features in the third key information area; determines whether the waybill image to be detected is a duplicate waybill based on the key text data, express attribute data, waybill graphic features corresponding to the waybill image to be detected and the preset database. The present invention can automatically, quickly and accurately identify duplicate waybills, avoid wrong or missed delivery of parcels, and ensure accurate delivery. Automated detection reduces manual review time, improves efficiency, reduces error costs, and avoids the cost of parcel loss, damage compensation, and customer complaint handling.

[0067] It should be noted that the order in which the embodiments of this specification are presented is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions are of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0068] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, equipment, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant portions, refer to the descriptions of the method embodiments.

[0069] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or by a program instructing the relevant hardware to accomplish the steps. The program may be stored in a computer storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for checking duplicate delivery of express delivery bills, characterized in that: The method comprises: Storing historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database; Obtain an image of the express parcel to be inspected, input the image into a face order region detection model to perform target detection, and identify key information regions in the face order image and the information category corresponding to each key information region; Acquire a first key information area whose information category is key text information, and extract key text data in the first key information area; Acquire a second key information area whose information category is barcode information, and extract express attribute data in the second key information area; Acquire a third key information region of the waybill image whose information category is information category, and extract the waybill graphic features in the third key information region; Determine whether the waybill image to be detected is a duplicate waybill based on the key text data, express attribute data, waybill graphic features and the preset database corresponding to the waybill image to be detected.

2. The method according to claim 1, characterized in that Before storing the historical key text data, historical express attribute data, and historical express graphic features corresponding to the historical express images in the preset database, the method further includes: Obtaining historical express delivery image of historical express parcels, and processing the historical delivery image to obtain a historical binary image; Using OCR technology to identify historical key text information in the historical binary image, and converting the historical key text information into historical key text data; Using a barcode recognition algorithm to extract historical barcode information from the historical binary image, and parsing the historical barcode information to obtain historical express attribute data of the historical express package; The historical face single graphic features are extracted from the historical binary image by using an image morphology algorithm.

3. The method according to claim 2, characterized in that The method of obtaining a historical express parcel's historical waybill image and processing the historical waybill image to obtain a historical binary image includes: Get historical express parcel waybill images; Grayscale processing is performed on the historical face image to obtain a historical grayscale image; Using a Gaussian filtering algorithm to remove noise from the historical grayscale image to obtain a historical denoised image; An adaptive threshold segmentation method is adopted to automatically determine a threshold according to the local grayscale characteristics of the historical denoised image, so as to separate the text and barcode in the historical denoised image from the background and obtain the historical binary image.

4. The method according to claim 1, wherein The determining whether the face order image to be detected is a duplicate face order based on key text data, express attribute data, face order graphic features, and the preset database corresponding to the face order image to be detected includes: Calculating a first similarity between each historical key text data in the preset database and the key text data; Determine the historical express delivery image corresponding to the historical key text data whose first similarity is greater than the first threshold as the screening express delivery image, and obtain the historical express delivery attribute data and the historical express delivery graphic features corresponding to the screening express delivery image as the screening express delivery attribute data and the screening express delivery graphic features; Calculating a second similarity between the express attribute data and the filtered express attribute data and a third similarity between the waybill graphic feature and the filtered waybill graphic feature; If the second similarity is greater than a second threshold and the third similarity is greater than a third threshold, it is determined that the face order image to be detected is a duplicate face order.

5. The method according to claim 4, characterized in that After determining that the face order image to be detected is a duplicate face order, the method further includes: Obtaining the image of the express package to be inspected, key text data, express attribute data, express label graphic features, express label number, and express label comparison results, generating duplicate express label information and triggering an alarm mechanism; the express label comparison results include the first similarity, the second similarity, and the third similarity; The duplicate bill of lading information is sent to the terminal of the auditing user so that the auditing user can verify the duplicate bill of lading information and take corrective measures to correct the duplicate bill of lading.

6. The method according to claim 1, characterized in that The training method of the face sheet area detection model includes: Obtain a sample face sheet image of a sample express package; the sample face sheet image is annotated with a sample key information area label and a sample information category label corresponding to the sample key information area; the sample key information area includes a sample key text information area, a sample barcode information area, and a sample face sheet image area; Input the sample face single image into the YOLOv8 model for region prediction and region information category prediction to obtain the sample key information region result and the sample information category result; determining target loss data based on a difference between the sample key and information region result and the sample key information region label, and a difference between the sample information category result and the sample information category label; The model parameters of the YOLOv8 model are adjusted based on the target loss data until the training end condition is met, and the YOLOv8 model at the end of training is determined as the face single area detection model.

7. The method according to claim 6, characterized in that The YOLOv8 model includes a backbone network, a neck network, and a head network. The sample face single image is input into the YOLOv8 model for region prediction and region information category prediction to obtain a sample key information region result and a sample information category result, including: Extracting image features and text features from the sample face image based on the backbone network; Based on the neck network, the image features and the text features are fused and enhanced to obtain sample area features; Based on the head network, regional prediction and regional information category prediction are performed on the sample region features to obtain sample key information region results and sample information category results.

8. A device for checking the duplicate of an express delivery bill, characterized in that: The device comprises: A historical waybill feature storage module is used to store historical key text data, historical express attribute data, and historical waybill graphic features corresponding to historical waybill images in a preset database; A key information area recognition module is used to obtain an image of the express parcel to be inspected, input the image into the parcel area detection model to perform target detection, and identify the key information areas in the parcel image and the information category corresponding to each key information area; A key text data extraction module, configured to obtain a first key information region whose information category is key text information, and extract key text data from the first key information region; an express attribute data extraction module, configured to obtain a second key information area whose information category is barcode information, and extract express attribute data from the second key information area; a waybill graphic feature extraction module, configured to obtain a third key information region of the waybill image whose information category is the waybill image, and extract the waybill graphic features in the third key information region; A duplicate bill determination module is used to determine whether the bill image to be detected is a duplicate bill based on the key text data, express attribute data, bill graphic features and the preset database corresponding to the bill image to be detected.

9. An electronic device, characterized in that: The device includes: a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the express delivery bill duplicate checking method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the express delivery bill duplicate checking method as described in any one of claims 1-7.

Images