Express sheet bar code identification method, apparatus and device, and storage medium

Abstract

The invention relates to the technical field of information processing, in particular to an express sheet bar code identification method, device and equipment and a storage medium, historical express sheet image data of all express service providers are collected, so that a data set meets the requirements of model training in scale and content richness, and the identification accuracy of the express sheet bar code is improved. The method comprises the steps of classifying express sheet image preprocessing data according to different sheet types and styles, carrying out data division on express sheet image classification data to obtain a training data set, labeling a region where a bar code in the training data set is located, building a deep learning model based on a DETR target detection algorithm, and carrying out deep learning on the deep learning model; and inputting the training data set and the bar code labeling information into a deep learning model for training to obtain a bar code recognition model, uploading the express sheet image information to the bar code recognition model for bar code recognition, and automatically detecting bar code areas of different sheet types and styles by using a target detection algorithm. And the identification accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method, device, equipment and storage medium for identifying a face sheet barcode. Background Art

[0002] With the booming development of e-commerce, the logistics industry has increasingly higher requirements for the efficiency and accuracy of express package processing. The barcode on the waybill is an important carrier of package information. Its accurate recognition is crucial to ensure the accurate delivery of goods. Traditional OCR technology has certain limitations in barcode recognition, especially in terms of image quality, barcode angle and position. The length, density, clarity, contrast and integrity of the barcode will affect the recognition rate of the barcode scanner. In addition, the barcodes in the waybill images of different types and styles are different, and barcodes of different types or styles may not be recognized. Summary of the Invention

[0003] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a face sheet barcode recognition method, device, equipment and storage medium that enables the data set to meet the requirements of model training in terms of scale and content richness, and can use a target detection algorithm to automatically detect barcode areas of different face sheet types and styles, thereby improving the recognition accuracy.

[0004] The first aspect of the present invention provides a method for identifying a waybill barcode, comprising: collecting historical express waybill image data of all express service providers, and performing data preprocessing on the historical express waybill image data to obtain express waybill image preprocessing data; classifying the express waybill image preprocessing data according to different waybill types and styles to obtain express waybill image classification data, and performing data division on the express waybill image classification data to obtain a training data set; marking the area where the barcode is located in the training data set to obtain barcode marking information, and verifying the barcode marking information to obtain a verification result; when the verification result is passed, building a deep learning model based on the DETR target detection algorithm, and inputting the training data set and the barcode marking information into the deep learning model for training to obtain a barcode recognition model; acquiring express waybill image information in real time, and uploading the express waybill image information to the barcode recognition model for barcode recognition to obtain a barcode recognition result.

[0005] Optionally, in a first implementation method of the first aspect of the present invention, the historical express delivery bill image data of all express delivery service providers is collected, and the historical express delivery bill image data is pre-processed to obtain the express delivery bill image pre-processed data, including: connecting with the API interfaces of all express delivery service providers respectively, and receiving the historical express delivery bill image encrypted data through the API interfaces of the express delivery service providers; decrypting the historical express delivery bill image encrypted data to obtain the historical express delivery bill image decrypted data; deduplicating the historical express delivery bill image decrypted data to obtain the historical express delivery bill image deduplication data; and performing format conversion on the historical express delivery bill image deduplication data based on a preset target standard format to obtain the express delivery bill image pre-processed data.

[0006] Optionally, in a second implementation method of the first aspect of the present invention, the express delivery bill image preprocessing data is classified according to different bill types and styles to obtain express delivery bill image classification data, and the express delivery bill image classification data is divided to obtain a training data set, including: classifying the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data; dividing the express delivery bill image classification data to obtain an initial training set, an initial validation set, and an initial test set; and using Cutout and mixup to perform data augmentation on the initial training set, the initial validation set, and the initial test set to obtain a training data set, a validation data set, and a test data set.

[0007] Optionally, in a third implementation method of the first aspect of the present invention, the area where the barcode is located in the training data set is marked to obtain barcode marking information, and the barcode marking information is verified to obtain a verification result, including: calling the LabelImg marking tool to import the training data set into the LabelImg marking tool; using the LabelImg marking tool to mark the area where the barcode is located in the training data set to obtain barcode marking information; and verifying the barcode marking information to obtain a verification result.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, when the verification result is passed, a deep learning model is built based on the DETR target detection algorithm, and the training data set and the barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model, including: when the verification result is failed, the barcode annotation information is corrected; when the verification result is passed, a Swin-Transformer image processing model is built, and the DETR target detection algorithm is introduced into the Swin-Transformer image processing model; the training data set and the barcode annotation information are input into the Swin-Transformer image processing model for training to obtain a barcode recognition model; and the barcode recognition model is tuned and evaluated using a verification data set and a test data set respectively.

[0009] Optionally, in a fifth implementation method of the first aspect of the present invention, the real-time acquisition of express delivery bill image information, uploading the express delivery bill image information to the barcode recognition model for barcode recognition, and obtaining a barcode recognition result includes: creating an express delivery bill image listener, and using the express delivery bill image listener to listen to the express delivery bill image information sent by the user terminal; when the express delivery bill image listener listens to the express delivery bill image information, obtaining the current express delivery bill image information; uploading the express delivery bill image information to the barcode recognition model for barcode recognition, and obtaining a barcode recognition result.

[0010] Optionally, in a sixth implementation method of the first aspect of the present invention, the real-time acquisition of express delivery bill image information, uploading the express delivery bill image information to the barcode recognition model for barcode recognition, and obtaining the barcode recognition result further includes: matching the express delivery order information corresponding to the barcode recognition result; generating express delivery order visualization information based on the express delivery order information; and sending the express delivery order visualization information to the user terminal, so that the user terminal generates and displays an express delivery order visualization page based on the express delivery order visualization information.

[0011] The second aspect of the present invention provides a waybill barcode recognition device, comprising: a collection and processing module for collecting historical express waybill image data of all express service providers, and performing data preprocessing on the historical express waybill image data to obtain express waybill image preprocessing data; a classification and division module for classifying the express waybill image preprocessing data according to different waybill types and styles to obtain express waybill image classification data, and performing data division on the express waybill image classification data to obtain a training data set; a labeling and verification module for labeling the area where the barcode is located in the training data set to obtain barcode labeling information, and verifying the barcode labeling information to obtain a verification result; a training module for building a deep learning model based on the DETR target detection algorithm when the verification result is passed, and inputting the training data set and the barcode labeling information into the deep learning model for training to obtain a barcode recognition model; an acquisition and recognition module for acquiring express waybill image information in real time, uploading the express waybill image information to the barcode recognition model for barcode recognition, and obtaining a barcode recognition result.

[0012] Optionally, in a first implementation method of the second aspect of the present invention, the collection and processing module includes: a docking and receiving unit, used to dock with the API interfaces of all express service providers respectively, and receive historical express delivery bill image encrypted data through the API interface of the express service provider; a decryption unit, used to decrypt the historical express delivery bill image encrypted data to obtain historical express delivery bill image decrypted data; a deduplication unit, used to deduplicate the historical express delivery bill image decrypted data to obtain historical express delivery bill image deduplication data; a conversion unit, used to convert the format of the historical express delivery bill image deduplication data based on a preset target standard format to obtain express delivery bill image pre-processing data.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the classification and division module includes: a classification unit, used to classify the express waybill image preprocessing data according to different waybill types and styles to obtain express waybill image classification data; a division unit, used to perform data division on the express waybill image classification data to obtain an initial training set, an initial verification set and an initial test set; an augmentation unit, used to perform data augmentation on the initial training set, the initial verification set and the initial test set using Cutout and mixup to obtain a training data set, a verification data set and a test data set.

[0014] Optionally, in a third implementation of the second aspect of the present invention, the annotation verification module includes: a calling import unit, used to call the LabelImg annotation tool to import the training data set into the LabelImg annotation tool; a annotation unit, used to use the LabelImg annotation tool to mark the area where the barcode in the training data set is located to obtain barcode annotation information; and a verification unit, used to verify the barcode annotation information to obtain a verification result.

[0015] Optionally, in a fourth implementation method of the second aspect of the present invention, the training module includes: a correction unit, used to correct the barcode annotation information when the verification result is failed; an introduction unit, used to build a Swin-Transformer image processing model when the verification result is passed, and introduce the DETR target detection algorithm into the Swin-Transformer image processing model; a training unit, used to input the training data set and the barcode annotation information into the Swin-Transformer image processing model for training to obtain a barcode recognition model; and a tuning and evaluation unit, used to tune and evaluate the barcode recognition model through a verification data set and a test data set respectively.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the acquisition and identification module includes: creating a monitoring unit, used to create an express delivery bill image monitor, and using the express delivery bill image monitor to monitor the express delivery bill image information sent by the user terminal; an acquisition unit, used to obtain the current express delivery bill image information when the express delivery bill image monitor monitors the express delivery bill image information; and an identification unit, used to upload the express delivery bill image information to the barcode recognition model for barcode recognition to obtain a barcode recognition result.

[0017] Optionally, in the sixth implementation method of the second aspect of the present invention, it also includes: a matching module for matching express order information corresponding to the barcode recognition result; a generation module for generating express order visualization information based on the express order information; and a sending module for sending the express order visualization information to the user terminal, so that the user terminal generates and displays an express order visualization page based on the express order visualization information.

[0018] The third aspect of the present invention provides a face label barcode recognition device, which includes: a memory and at least one processor, wherein the memory stores instructions; at least one processor calls the instructions in the memory to enable the face label barcode recognition device to execute each step of the face label barcode recognition method described above.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of any of the above-mentioned methods for identifying the face sheet barcode.

[0020] In the technical solution of the present invention, historical express delivery bill image data of all express delivery service providers is collected so that the data set meets the requirements of model training in terms of scale and content richness, the express delivery bill image preprocessing data is classified according to different delivery bill types and styles, the express delivery bill image classification data is divided to obtain a training data set, the area where the barcode is located in the training data set is marked, and a deep learning model is built based on the DETR target detection algorithm. The training data set and barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model, and the express delivery bill image information is uploaded to the barcode recognition model for barcode recognition. The target detection algorithm can be used to automatically detect the barcode areas of different delivery bill types and styles, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A first flow chart of a method for identifying a label barcode provided by an embodiment of the present invention;

[0022] Figure 2 A second flow chart of the method for identifying a label barcode provided by an embodiment of the present invention;

[0023] Figure 3 A third flow chart of the method for identifying a label barcode provided by an embodiment of the present invention;

[0024] Figure 4 A fourth flow chart of the method for identifying a label barcode provided by an embodiment of the present invention;

[0025] Figure 5 A schematic structural diagram of a label barcode recognition device provided by an embodiment of the present invention;

[0026] Figure 6 Another structural diagram of the invoice barcode recognition device provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a label barcode recognition device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention provides a method, apparatus, device and storage medium for identifying face sheet barcodes, which enables the dataset to meet the requirements of model training in terms of scale and content richness, and can use a target detection algorithm to automatically detect barcode areas of different face sheet types and styles, thereby improving recognition accuracy.

[0029] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 In one embodiment of the present invention, a method for identifying a barcode on a sheet includes:

[0031] 101. Collect historical express delivery bill image data from all express delivery service providers, and perform data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data;

[0032] In this embodiment, multiple express service providers (such as SF Express, STO Express, YTO Express, etc.) are connected to obtain historical express delivery bill image data of the express service providers. The data provided by these express service providers can be obtained through the API interface. The data usually includes image files (delivery bill images). These images may be scanned pictures or digitized files (such as PDF, PNG, JPG, etc. formats). If the delivery bill data provided by the express service provider is encrypted, it needs to be decrypted. The image hash (such as MD5, SHA) is used to check whether there are duplicate image files, or the image content is compared to identify duplications. The duplicate historical express delivery bill image data is deleted, and finally the historical express delivery bill image data is unified into a specific format to obtain express delivery bill image preprocessing data.

[0033] 102. Classify the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data, and divide the express delivery bill image classification data to obtain a training data set;

[0034] In this embodiment, each express delivery company may have different styles and designs. These differences may be reflected in color, layout, font, information typesetting, etc. The definition of each waybill style can be distinguished based on the visual features of the image (such as color, font size, barcode position, etc.), and then the waybill type can be identified. For each waybill type, it is necessary to extract the unique visual features of the type and label each waybill image. The label may include the express delivery company name, waybill type, waybill style, information area location, etc. Through the previous classification process, multiple different categories of waybill images will be obtained. These images will be divided into different categories according to different waybill types and styles. Each category represents a waybill of an express delivery company or a style. Once the classification of the waybill images is completed, the next step is to divide these data into training data set, verification data set and test data set.

[0035] 103. Annotate the barcode region in the training data set to obtain barcode annotation information, and verify the barcode annotation information to obtain a verification result.

[0036] In this embodiment, in the training data set, the barcode may appear in different positions on the face sheet and may have different sizes, angles or shapes. A rectangular frame is used to mark the barcode area. The marked rectangular frame needs to accurately cover the entire area of the barcode, including all lines and spaces of the barcode. Coordinates are used in the image to represent the position of the marked area. A corresponding label is added to each marked area to identify the area as a barcode area. The label is generally "Barcode". The marking information of each face sheet is checked to ensure the accuracy and consistency of the marking. Images with incorrect or unclear markings are corrected or re-marked in a timely manner to lay the foundation for subsequent high-quality model training. During verification, first check whether the marked frame completely covers the barcode area to ensure that no part of the barcode is missed, ensure that each barcode is marked only once, and no repeated marking occurs. The marked rectangular frame should fit closely to the edge of the barcode to avoid being too large or too small, and ensure that the selected area conforms to the actual size of the barcode to finally obtain the verification result.

[0037] 104. If the verification result is passed, a deep learning model is built based on the DETR target detection algorithm, and the training data set and barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model;

[0038] In this embodiment, when the verification result is passed, a Swin-Transformer image processing model is built, a DEtection TRansformer (DETR) module is added to the Swin-Transformer image processing model, and the training data set and barcode annotation information are input into the Swin-Transformer image processing model for training to obtain a barcode recognition model.

[0039] 105. Acquire the express delivery bill image information in real time, upload the express delivery bill image information to the barcode recognition model for barcode recognition, and obtain the barcode recognition result;

[0040] In this embodiment, the express delivery bill image information is obtained in real time, and the express delivery bill image information is uploaded to the barcode recognition model for barcode recognition. The feature map output by the barcode recognition model, the DETR module is mainly composed of two parts: the Transformer encoder and the decoder. First, the feature map output by the barcode recognition model is input into the DETR encoder. The encoder further encodes the feature map through the self-attention mechanism, captures the global dependency between different positions in the feature map, so that the feature information can interact and fuse in a larger range, and strengthens the learning of the associated features between the barcode area and other areas in the express delivery bill image. Then, the encoded feature information is passed to the DETR decoder. The decoder will interact with the encoded features based on a set of learnable target query vectors to directly predict the category of the target (i.e., the barcode on the express delivery bill) and its position information in the image (expressed in the form of bounding box coordinates) to obtain the barcode recognition result.

[0041] In an embodiment of the present invention, historical express delivery bill image data of all express delivery service providers is collected so that the data set meets the requirements of model training in terms of scale and content richness, the express delivery bill image preprocessing data is classified according to different delivery bill types and styles, the express delivery bill image classification data is divided to obtain a training data set, the area where the barcode is located in the training data set is marked, and a deep learning model is built based on the DETR target detection algorithm. The training data set and barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model, and the express delivery bill image information is uploaded to the barcode recognition model for barcode recognition. The target detection algorithm can be used to automatically detect the barcode areas of different delivery bill types and styles, thereby improving the recognition accuracy.

[0042] See also Figure 2 The second embodiment of the method for identifying a barcode on a sheet of paper according to the present invention includes:

[0043] 201. Connect to the API interfaces of all express service providers respectively, and receive encrypted image data of historical express delivery orders through the API interfaces of the express service providers;

[0044] In this embodiment, an adapter (or API client) is developed based on the API documentation of each express service provider to achieve docking with the API interface of each service provider. According to the API interface documentation of each express service provider, the correct API request method (usually a GET request) is used to receive historical express delivery order image data. Parameters such as express delivery order number, time range, batch number, etc. need to be specified when making the request. Some service providers may support conditional query of historical data (for example, querying all express delivery order number information within a certain time period), thereby facilitating the acquisition of specific express delivery order image data. The API interface of the express service provider will return encrypted express delivery order image data.

[0045] 202. Decrypt the encrypted image data of the historical express delivery waybill to obtain the decrypted image data of the historical express delivery waybill;

[0046] In this embodiment, the received historical express delivery bill image encrypted data is usually in Base64 encoding format. Before decryption, it is first necessary to decode the Base64-encoded historical express delivery bill image encrypted data back into binary data, that is, to parse the historical express delivery bill image encrypted data. After parsing, according to the encryption standard (such as AES or RSA), the decoded binary data is decrypted using the corresponding key to restore it to the original image data, thereby obtaining the historical express delivery bill image decrypted data.

[0047] 203. Deduplication processing is performed on the decrypted data of the historical express delivery bill images to obtain deduplication data of the historical express delivery bill images;

[0048] In this embodiment, a hash value is generated for the image file and each image is mapped to a unique hash value. For the same image, the hash value is the same, while for different images, the hash value will also be different. The hash value is calculated for all images, and images with the same hash value are marked as duplicates. Finally, the duplicate images are removed to obtain the deduplicated data of historical express delivery order images.

[0049] 204. Based on a preset target standard format, perform format conversion on the deduplication data of historical express delivery bill images to obtain express delivery bill image pre-processing data;

[0050] In this embodiment, the format of the deduplicated data of historical express delivery bill images is converted to a predetermined target standard format, such as PNG, JPEG, etc., to obtain express delivery bill image preprocessing data.

[0051] 205. Classify the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data;

[0052] In this embodiment, the definition of each waybill style is distinguished based on the visual features of the image (such as color, font size, barcode position, etc.), and then the waybill type is identified. For each waybill type, the unique visual features of the type are extracted, and each waybill image is labeled. The labels include the courier company name, waybill type, waybill style, information area location, etc. Through the previous classification process, multiple waybill images of different categories will be obtained. These images will be divided into different categories according to different waybill types and styles. Each category represents a courier company's waybill or a style. Once the classification of the waybill image is completed, the express waybill image classification data is obtained.

[0053] 206. Divide the express delivery label image classification data into an initial training set, an initial validation set, and an initial test set;

[0054] In this embodiment, the express delivery label image classification data is divided into data in proportion to obtain an initial training set, an initial validation set, and an initial test set. The initial training set is used to train the model, accounting for 70% to 80% of the total express delivery label image classification data. The initial validation set is used to tune the hyperparameters of the model and select the best model, accounting for 10% to 15% of the total express delivery label image classification data. The initial test set is used to evaluate the performance of the model, accounting for 10% to 15% of the total express delivery label image classification data.

[0055] 207. Use Cutout and Mixup to perform data augmentation on the initial training set, initial validation set, and initial test set to obtain the training data set, validation data set, and test data set;

[0056] In this embodiment, AutoAugment is used to perform the following basic transformations on the images of the initial training set, initial validation set, and initial test set: horizontally flipping the image, randomly rotating it by a certain angle, scaling the image proportionally, randomly translating the image in the horizontal or vertical direction, and randomly changing the brightness of the image. After applying the above enhancement strategy, the first newly added initial training set, initial validation set, and initial test set will contain images that have been mirrored, rotated, scaled, translated, and brightness adjusted, while maintaining the original state of the training set. Then, the size and number of the occlusion area are set, and Cutout is used to randomly select a rectangular area on the images of the first newly added initial training set, initial validation set, and initial test set and block it out to achieve the first new The added initial training set, initial validation set and initial test set are cropped while keeping the original state of the first added initial training set, initial validation set and initial test set. After applying Cutout, the second added initial training set, initial validation set and initial test set generated will contain the cropped images. A hyperparameter is set to control the intensity of sample mixing. Mixup is applied to the initial training set, initial validation set and initial test set, the first added initial training set, initial validation set and initial test set and the second added initial training set, initial validation set and initial test set. DataLoader is used to load and mix the data to generate an augmented version for each data set to obtain the training data set, validation data set and test data set.

[0057] In the embodiment of the present invention, the quality and classification accuracy of the waybill image are effectively improved through systematic data processing and augmentation steps. First, by connecting to the API interfaces of multiple express service providers, the diversity and comprehensiveness of the data are ensured. Steps such as decryption, deduplication, and format conversion provide clear and standardized input for subsequent data processing, thereby improving the consistency and reliability of the data. Classification and data partitioning ensure the efficiency of model training, and the diversity of the training set is enhanced through data augmentation technology, which helps to improve the generalization ability and robustness of the model.

[0058] See also Figure 3 The third embodiment of the method for identifying a barcode on a sheet of paper according to the present invention includes:

[0059] 301. Call the LabelImg annotation tool and import the training data set into the LabelImg annotation tool;

[0060] In this embodiment, the LabelImg labeling tool is installed and configured in a local or cloud environment, and the training dataset is imported into the LabelImg labeling tool.

[0061] 302. Label the barcode area in the training data set using the LabelImg labeling tool to obtain barcode labeling information;

[0062] In this embodiment, a separate label, such as "Barcode," is created for the barcode category. A rectangular box is drawn to enclose the barcode area to form a label. The labeled rectangular box must accurately cover the entire barcode area, including all lines and spaces of the barcode. Coordinates are used in the image to represent the location of the labeled area. A corresponding label is added to each labeled area to identify it as a barcode area.

[0063] 303. Verify the barcode label information to obtain a verification result;

[0064] In this embodiment, the barcode marking information of each label is checked to ensure the accuracy and consistency of the marking, and images with incorrect or unclear markings are corrected or re-marked in a timely manner to lay the foundation for subsequent high-quality model training. During the verification, it is first necessary to check whether the marking box completely covers the barcode area to ensure that no part of the barcode is missed, and to ensure that each barcode is marked only once without repeated markings. The marked rectangular box should fit closely to the edge of the barcode to avoid being too large or too small, and to ensure that the selected area conforms to the actual size of the barcode to finally obtain the verification result.

[0065] 304. When the verification result is not passed, the barcode labeling information is corrected;

[0066] In this embodiment, when the verification result is failure, the specific error type is identified, and the barcode annotation information is corrected according to the marked error type.

[0067] 305. If the verification result is passed, a Swin-Transformer image processing model is built and the DETR target detection algorithm is introduced into the Swin-Transformer image processing model;

[0068] In this embodiment, when the verification result is passed, the Swin-Transformer framework is selected, the model architecture is defined, and the DETR module is integrated into the Swin-Transformer framework. Specifically, the Swin-Transformer framework is used as the feature extractor of DETR, and the output of Swin-Transformer is used as the input of the DETR encoder.

[0069] 306. Input the training data set and the barcode annotation information into the Swin-Transformer image processing model for training to obtain a barcode recognition model;

[0070] In this embodiment, the training data set and the barcode annotation information are preprocessed, and then the training data set and the barcode annotation information are input into a Swin-Transformer image processing model for training to obtain a barcode recognition model.

[0071] 307. Fine-tune and evaluate the barcode recognition model using the validation dataset and test dataset respectively;

[0072] In this embodiment, a validation dataset is used to adjust the hyperparameters of the barcode recognition model, such as the learning rate, batch size, regularization method, and number of training rounds. For example, the validation dataset is used to detect whether the learning rate is appropriate and whether the batch size is too large or too small. In this process, methods such as grid search or random search can be used to find the optimal combination. During the tuning process, the main role of the validation dataset is to help prevent the barcode recognition model from overfitting on the training dataset, and the test dataset is used to evaluate the final performance of the barcode recognition model.

[0073] In the embodiment of the present invention, data quality is ensured through a sophisticated labeling and verification process, and the accuracy and efficiency of barcode recognition are improved by combining advanced Swin-Transformer and DETR algorithms. The labeling and verification steps ensure the accuracy of the training data and provide a reliable foundation for subsequent model training. During the model training process, advanced target detection technology is combined to enhance the model's recognition ability in complex image environments. Through the entire process of training, tuning and evaluation, the efficiency and reliability of the model are ensured, and the practical application value of the model is improved through continuous optimization.

[0074] See also Figure 4 The fourth embodiment of the method for identifying a barcode on a sheet of paper according to the present invention includes:

[0075] 401. Create an express delivery bill image listener and use the express delivery bill image listener to monitor the express delivery bill image information sent by the user terminal;

[0076] In this embodiment, an express delivery bill image listener is created. The express delivery bill image listener receives the express delivery bill image from the user end through a network interface (such as HTTP, WebSocket, etc.). The user end may be a mobile application, web page or other system. The user can take or upload the express delivery bill image to the server.

[0077] 402. When the express delivery bill image monitor monitors the express delivery bill image information, the current express delivery bill image information is obtained;

[0078] In this embodiment, the express delivery bill image listener is in a real-time monitoring state and is ready to receive the express delivery bill image uploaded by the user at any time. When the user uploads the express delivery bill image, the express delivery bill image listener will be triggered. The listener continuously monitors the upload interface so as to receive the image information in time when the image is uploaded. Once the express delivery bill image listener receives the image data, it needs to perform preliminary confirmation to ensure the integrity and validity of the data.

[0079] 403. Upload the express delivery label image information to the barcode recognition model for barcode recognition, and obtain a barcode recognition result;

[0080] In this embodiment, the express delivery label image information is uploaded to the barcode recognition model for barcode recognition. Swin-Transformer is used to extract features of the input image to obtain global and local features of the image. These features will be used as input to the DETR model. The features extracted by Swin-Transformer are input to the encoder of DETR to generate the feature representation required for target detection. Then, the decoder of DETR gradually predicts the barcode information in the image, such as category labels and bounding box coordinates, through the self-attention mechanism to obtain the barcode recognition result.

[0081] 404. Match the express order information corresponding to the barcode recognition result;

[0082] In this embodiment, once the barcode recognition result (such as the waybill number or order number) is obtained, the next step is to query the system database for the express order information corresponding to the barcode data. The system searches for matching records in the database based on the order number or waybill number obtained from the barcode recognition model. The system compares the recognized barcode data with all order records in the database. If the waybill number or order number in the barcode is consistent with the order information in the database, it is considered a successful match. Once the match is successful, the corresponding logistics tracking information is generated based on the order number or waybill number.

[0083] 405. Generate express order visualization information based on the express order information;

[0084] In this embodiment, based on the extracted order information, an appropriate visual display method is designed to facilitate user understanding and operation. The various stages of the order from shipment to delivery are displayed through a progress bar or segmented diagram. For example, the progress bar can be divided into stages such as "shipped", "in transit", "arrived at destination", and "delivered". Each stage can be equipped with a corresponding date or timestamp. The basic information of the order (such as sender, recipient, waybill number, cargo weight, etc.) is displayed in the form of a card or panel. Each item of information can be distinguished by a different color or icon to enhance the visualization effect. An interactive map is used to display the current location of the package and the past transportation route. Each important transfer point and time point can be marked on the route. A countdown display is added to the order display interface to indicate how many hours or days are left until the estimated delivery time.

[0085] 406. Sending the express order visualization information to the user terminal, so that the user terminal generates and displays the express order visualization page based on the express order visualization information;

[0086] In this embodiment, the express order visualization information is sent to the user end, and the user end dynamically generates and displays an express order visualization page with a good user experience. The express order visualization page includes basic order information, logistics progress bar, transportation route map, etc.

[0087] In the embodiment of the present invention, efficient recognition and accurate matching of express delivery bill information are ensured through automated image monitoring and real-time processing. By combining barcode recognition technology with order information, visual data of the order can be quickly generated and transmitted, improving user experience and operational efficiency. The automation of the entire process and the rapid response of information flow ensure smooth and efficient processing from bill image acquisition to final display, reducing manual intervention and optimizing the interactive experience on the user side.

[0088] The above describes the method for identifying the barcode of the sheet in the embodiment of the present invention. The following describes the device for identifying the barcode of the sheet in the embodiment of the present invention. Figure 5 In one embodiment of the present invention, a device for identifying a barcode on a sheet includes:

[0089] The collection and processing module 501 is used to collect historical express delivery bill image data from all express delivery service providers and perform data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data;

[0090] A classification and partitioning module 502 is configured to classify the pre-processed express delivery bill image data according to different bill types and styles to obtain classified express delivery bill image data, and to partition the classified express delivery bill image data to obtain a training data set;

[0091] The marking and verification module 503 is used to mark the barcode area in the training data set to obtain barcode marking information, and to verify the barcode marking information to obtain a verification result;

[0092] Building a training module 504, which is used to build a deep learning model based on the DETR target detection algorithm when the verification result is passed, and input the training data set and barcode annotation information into the deep learning model for training to obtain a barcode recognition model;

[0093] The acquisition and recognition module 505 is used to obtain the express delivery bill image information in real time, upload the express delivery bill image information to the barcode recognition model for barcode recognition, and obtain the barcode recognition result.

[0094] In this embodiment, historical express delivery bill image data of all express delivery service providers is collected so that the data set meets the requirements of model training in terms of scale and content richness. The express delivery bill image preprocessing data is classified according to different bill types and styles, and the express delivery bill image classification data is divided to obtain a training data set. The barcode area in the training data set is labeled, and a deep learning model is built based on the DETR target detection algorithm. The training data set and barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model. The express delivery bill image information is uploaded to the barcode recognition model for barcode recognition. The target detection algorithm can be used to automatically detect the barcode areas of different bill types and styles, thereby improving the recognition accuracy.

[0095] See also Figure 6 Another embodiment of the apparatus for identifying a barcode on a sheet according to the present invention includes:

[0096] The collection and processing module 501 is used to collect historical express delivery bill image data from all express delivery service providers and perform data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data;

[0097] A classification and partitioning module 502 is configured to classify the pre-processed express delivery bill image data according to different bill types and styles to obtain classified express delivery bill image data, and to partition the classified express delivery bill image data to obtain a training data set;

[0098] The marking and verification module 503 is used to mark the barcode area in the training data set to obtain barcode marking information, and to verify the barcode marking information to obtain a verification result;

[0099] Building a training module 504, which is used to build a deep learning model based on the DETR target detection algorithm when the verification result is passed, and input the training data set and barcode annotation information into the deep learning model for training to obtain a barcode recognition model;

[0100] The acquisition and recognition module 505 is used to obtain the express delivery bill image information in real time, upload the express delivery bill image information to the barcode recognition model for barcode recognition, and obtain the barcode recognition result;

[0101] In this embodiment, the collection and processing module 501 includes: a docking and receiving unit 5011, which is used to dock with the API interfaces of all express service providers respectively, and receive historical express delivery bill image encrypted data through the API interface of the express service provider; a decryption unit 5012, which is used to decrypt the historical express delivery bill image encrypted data to obtain historical express delivery bill image decrypted data; a deduplication unit 5013, which is used to deduplicate the historical express delivery bill image decrypted data to obtain historical express delivery bill image deduplication data; a conversion unit 5014, which is used to convert the format of the historical express delivery bill image deduplication data based on a preset target standard format to obtain express delivery bill image pre-processing data.

[0102] In this embodiment, the classification and division module 502 includes: a classification unit 5021, which is used to classify the express delivery bill image pre-processing data according to different bill types and styles to obtain express delivery bill image classification data; a division unit 5022, which is used to perform data division on the express delivery bill image classification data to obtain an initial training set, an initial verification set and an initial test set; an augmentation unit 5023, which is used to use Cutout and mixup to perform data augmentation on the initial training set, the initial verification set and the initial test set to obtain a training data set, a verification data set and a test data set.

[0103] In this embodiment, the annotation verification module 503 includes: a calling import unit 5031, used to call the LabelImg annotation tool and import the training data set into the LabelImg annotation tool; a labeling unit 5032, used to use the LabelImg annotation tool to mark the area where the barcode is located in the training data set to obtain barcode annotation information; a verification unit 5033, used to verify the barcode annotation information and obtain a verification result.

[0104] In this embodiment, the training module 504 includes: a correction unit 5041, which is used to correct the barcode annotation information when the verification result is failed; an introduction unit 5042, which is used to build a Swin-Transformer image processing model when the verification result is passed, and introduce the DETR target detection algorithm into the Swin-Transformer image processing model; a training unit 5043, which is used to input the training data set and the barcode annotation information into the Swin-Transformer image processing model for training to obtain a barcode recognition model; and a tuning and evaluation unit 5044, which is used to tune and evaluate the barcode recognition model using a verification data set and a test data set respectively.

[0105] In this embodiment, the acquisition and identification module 505 includes: creating a monitoring unit 5051, which is used to create an express delivery bill image monitor, and use the express delivery bill image monitor to monitor the express delivery bill image information sent by the user terminal; an acquisition unit 5052, which is used to obtain the current express delivery bill image information when the express delivery bill image monitor monitors the express delivery bill image information; and an identification unit 5053, which is used to upload the express delivery bill image information to the barcode recognition model for barcode recognition to obtain the barcode recognition result.

[0106] In this embodiment, it also includes: a matching module 506, which is used to match the express order information corresponding to the barcode recognition result; a generating module 507, which is used to generate express order visualization information based on the express order information; and a sending module 508, which is used to send the express order visualization information to the user terminal, so that the user terminal generates and displays the express order visualization page based on the express order visualization information.

[0107] above Figure 5 and Figure 6 The face label barcode recognition device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The face label barcode recognition device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0108] Figure 7: This is a structural diagram of a face label barcode recognition device provided by an embodiment of the present invention. The face label barcode recognition device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 can be temporary storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the face label barcode recognition device 600. Furthermore, the processor 610 can be configured to communicate with the storage medium 630, and execute a series of instruction operations in the storage medium 630 on the face label barcode recognition device 600 to implement the steps of the face label barcode recognition method provided by the above-mentioned method embodiments.

[0109] The label barcode recognition device 600 may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 7 The illustrated structure of the face label barcode recognition device does not constitute a limitation on the face label barcode recognition device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0110] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the face sheet barcode recognition method.

[0111] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0113] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for identifying a barcode on a face sheet, characterized in that: include: Collect historical express delivery bill image data from all express delivery service providers, and perform data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data; Classifying the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data, and performing data division on the express delivery bill image classification data to obtain a training data set; Annotating the barcode region in the training data set to obtain barcode annotation information, and verifying the barcode annotation information to obtain a verification result; When the verification result is passed, a deep learning model is built based on the DETR target detection algorithm, and the training data set and the barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model; The express delivery bill image information is acquired in real time, and the express delivery bill image information is uploaded to the barcode recognition model for barcode recognition to obtain a barcode recognition result.

2. The method for identifying a label barcode according to claim 1, wherein: The collecting of historical express delivery bill image data of all express delivery service providers and performing data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data includes: Connect to the API interfaces of all express service providers respectively, and receive the encrypted image data of historical express delivery orders through the API interfaces of the express service providers; Decrypting the encrypted data of the historical express delivery bill image to obtain decrypted data of the historical express delivery bill image; Deduplication processing is performed on the decrypted data of the historical express delivery bill image to obtain deduplication data of the historical express delivery bill image; Based on a preset target standard format, the deduplication data of the historical express delivery bill images is format converted to obtain express delivery bill image preprocessing data.

3. The method for identifying a label barcode according to claim 1, wherein: The express delivery bill image preprocessing data is classified according to different bill types and styles to obtain express delivery bill image classification data, and the express delivery bill image classification data is divided into data to obtain a training data set, including: Classify the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data; Dividing the express delivery image classification data into an initial training set, an initial validation set, and an initial test set; Cutout and mixup are used to perform data augmentation on the initial training set, the initial validation set, and the initial test set to obtain a training data set, a validation data set, and a test data set.

4. The method for identifying a label barcode according to claim 1, wherein: The marking of the barcode region in the training data set to obtain barcode marking information, and the verification of the barcode marking information to obtain a verification result, include: Calling the LabelImg annotation tool and importing the training data set into the LabelImg annotation tool; Using the LabelImg annotation tool to annotate the barcode area in the training data set to obtain barcode annotation information; The barcode labeling information is verified to obtain a verification result.

5. The method for identifying a label barcode according to claim 1, wherein: When the verification result is passed, a deep learning model is built based on the DETR target detection algorithm, and the training data set and the barcode annotation information are input into the deep learning model for training to obtain a barcode recognition model, including: When the verification result is unsatisfactory, the barcode labeling information is corrected; When the verification result is passed, a Swin-Transformer image processing model is built, and a DETR target detection algorithm is introduced into the Swin-Transformer image processing model; Inputting the training data set and the barcode annotation information into the Swin-Transformer image processing model for training to obtain a barcode recognition model; The barcode recognition model was tuned and evaluated using a validation dataset and a test dataset, respectively.

6. The method for identifying a label barcode according to claim 1, wherein: The real-time acquisition of the express delivery bill image information, uploading the express delivery bill image information to the barcode recognition model for barcode recognition, and obtaining the barcode recognition result includes: Create an express delivery order image listener, and use the express delivery order image listener to monitor the express delivery order image information sent by the user terminal; When the express delivery bill image monitor monitors the express delivery bill image information, the current express delivery bill image information is obtained; The express delivery bill image information is uploaded to the barcode recognition model for barcode recognition to obtain a barcode recognition result.

7. The method for identifying a label barcode according to claim 1, wherein: The method further includes: acquiring the express delivery bill image information in real time, uploading the express delivery bill image information to the barcode recognition model for barcode recognition, and obtaining the barcode recognition result; Matching the express order information corresponding to the barcode recognition result; Generate express order visualization information according to the express order information; The express order visualization information is sent to a user terminal, so that the user terminal generates and displays an express order visualization page based on the express order visualization information.

8. A barcode recognition device for a single sheet, characterized in that: include: A collection and processing module is used to collect historical express delivery bill image data from all express delivery service providers, and perform data preprocessing on the historical express delivery bill image data to obtain express delivery bill image preprocessing data; a classification and partitioning module, configured to classify the express delivery bill image preprocessing data according to different bill types and styles to obtain express delivery bill image classification data, and perform data partitioning on the express delivery bill image classification data to obtain a training data set; a marking and verification module, configured to mark the barcode area in the training data set to obtain barcode marking information, and verify the barcode marking information to obtain a verification result; Building a training module for building a deep learning model based on the DETR target detection algorithm when the verification result is passed, and inputting the training data set and the barcode annotation information into the deep learning model for training to obtain a barcode recognition model; The acquisition and recognition module is used to obtain the express delivery bill image information in real time, upload the express delivery bill image information to the barcode recognition model for barcode recognition, and obtain the barcode recognition result.

9. A face-invoice barcode recognition device, characterized in that: The face sheet barcode recognition device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors calls the instructions in the memory to enable the face label barcode recognition device to perform each step of the face label barcode recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the face sheet barcode recognition method as described in any one of claims 1 to 7 are implemented.

Images