Express label identification method and device, computer device and storage medium

Abstract

This application provides a method, apparatus, computer device, and storage medium for recognizing express delivery waybills. The method includes: acquiring an image of the express delivery to be recognized; performing waybill region detection on the express delivery image to obtain a first rotation angle of a first waybill region detection frame and distance identification information (distances from target pixels in the first waybill region detection frame to each detection border of the first waybill region detection frame); determining the border correspondence between each detection border of the first waybill region detection frame and each actual border of the actual waybill image based on the distance identification information; correcting the image within the first waybill region detection frame based on the border correspondence and the first rotation angle to obtain a second waybill region detection frame; and recognizing the image within the second waybill region detection frame to obtain a waybill recognition result. This application can improve the success rate of waybill recognition.

Description

Technical Field

[0001] This application mainly relates to the field of express mail identification technology, specifically to an express mail waybill identification method, device, computer equipment, and storage medium. Background Technology

[0002] With the rapid development of internet technology and the gradual improvement of people's living standards, the express delivery and logistics industry has continued to expand, and the volume of express delivery business has grown rapidly year by year, exceeding 80 billion pieces in 2020. Behind this massive volume of express deliveries lies the strong support of logistics information technology. Decoding the waybill number from the logistics waybill is the first and most crucial step in logistics informatization. Currently, express waybills generally encode the waybill number information in barcode or QR code format. To ensure logistics sorting efficiency, sorting lines generally use specially designed hardware decoding systems to decode the waybill number. This hardware decoding system has a fast decoding speed, but it also has problems such as poor environmental adaptability and low decoding success rate in some scenarios. Common target detection algorithms, such as the EAST text detection algorithm, can obtain a rotated rectangle of the waybill area, but the rotated rectangle information of the EAST text detection algorithm cannot determine the orientation of the text. The waybill after rotation angle correction based on the output of the EAST text detection algorithm may be 90° or 180° inverted, resulting in a low success rate of express waybill recognition.

[0003] In other words, the success rate of recognizing express waybills in existing technologies is not high. Summary of the Invention

[0004] This application provides a method, apparatus, computer equipment, and storage medium for recognizing express delivery waybills, aiming to solve the problem of low success rate of express delivery waybill recognition in the prior art.

[0005] Firstly, this application provides a method for recognizing express delivery waybills, the method comprising:

[0006] Obtain an image of the package to be identified;

[0007] The package image is subjected to label area detection to obtain the first rotation angle of the first label area detection frame and the distance identification information of the target pixel in the first label area detection frame to each detection border of the first label area detection frame;

[0008] The correspondence between each detection border of the first label area detection box and each actual border of the actual label image is determined based on the distance identification information.

[0009] Based on the border correspondence and the first rotation angle, the image within the first single-area detection box is corrected to obtain the second single-area detection box.

[0010] The image within the second label area detection box is recognized to obtain the label recognition result.

[0011] Optionally, the step of recognizing the image within the second label region detection box to obtain the label recognition result includes:

[0012] The image within the second single-sided area detection box is input into the preset information area recognition model to obtain the predicted probability map of the preset information area within the second single-sided area detection box, wherein the preset information area includes at least one of a barcode area, a QR code area, and a text area;

[0013] The predicted probability map is binarized based on the predicted probability map of the preset information region and the preset threshold map to obtain a binarized mask map.

[0014] The preset information region is obtained by finding connected regions on the binarized mask image.

[0015] The image within the preset information area is recognized to obtain the waybill recognition result.

[0016] Optionally, the preset information area includes a barcode area, a QR code area, and a text area;

[0017] The step of recognizing the image within the preset information area to obtain the waybill recognition result includes:

[0018] Decode the barcode area to obtain first decoded information;

[0019] Decode the QR code area to obtain the second decoded information;

[0020] If neither the first decoding information nor the second decoding information includes the waybill number, then the waybill number is obtained by performing waybill recognition on the text area.

[0021] The waybill number is identified as the recognition result.

[0022] Optionally, the first rotation angle is the angle between the detection lower border of the first single-area detection frame and the horizontal axis;

[0023] The step of correcting the image within the first single-region detection box based on the border correspondence and the first rotation angle to obtain the second single-region detection box includes:

[0024] The second rotation angle is determined based on the corresponding border relationship;

[0025] The target rotation angle is determined based on the first rotation angle and the second rotation angle;

[0026] The image within the first single-area detection box is rotated based on the target rotation angle to obtain the second single-area detection box.

[0027] Optionally, the step of performing label region detection on the express delivery image to obtain the first rotation angle of the first label region detection frame and the distance identification information of the distance from the target pixel point in the first label region detection frame to each detection border of the first label region detection frame includes:

[0028] A preset waybill area detection model is obtained. The preset waybill area detection model is trained on a preset express delivery image sample set. The preset express delivery image sample set includes multiple express delivery sample images. The express delivery sample images are marked with waybill area annotation boxes and distance values ​​from each pixel in the waybill area annotation box to the four annotation borders of the waybill area annotation box and corresponding annotation labels. The annotation labels are determined according to the correspondence between the waybill area annotation box and each actual border of the actual waybill image.

[0029] The package image is input into the preset shipping label area detection model to obtain the first rotation angle of the first shipping label area detection frame and the distance identification information of the target pixel in the first shipping label area detection frame to each detection border of the first shipping label area detection frame.

[0030] Optionally, the distance identification information is a positive or negative sign of each distance from the target pixel to each detection border of the first single-area detection box.

[0031] Optionally, acquiring the image of the package to be identified includes:

[0032] The package to be identified is scanned to determine whether the scan information has been obtained.

[0033] If the barcode information of the package to be identified is not obtained, the package to be identified is photographed to obtain the image of the package.

[0034] Secondly, this application provides a waybill recognition device for express mail, the waybill recognition device comprising:

[0035] The acquisition unit is used to acquire images of the express package to be identified.

[0036] The waybill area detection unit is used to perform waybill area detection on the express image to obtain the first rotation angle of the first waybill area detection frame and the distance identification information of the target pixel point in the first waybill area detection frame to each detection border of the first waybill area detection frame.

[0037] The determining unit is used to determine the correspondence between each detection border of the first label area detection box and each actual border of the actual label image based on the distance identification information.

[0038] The correction unit is used to correct the image within the first single-area detection box based on the border correspondence and the first rotation angle to obtain the second single-area detection box.

[0039] The recognition unit is used to recognize the image within the second label area detection box to obtain the label recognition result.

[0040] Optionally, the identification unit is used for:

[0041] The image within the second single-sided area detection box is input into the preset information area recognition model to obtain the predicted probability map of the preset information area within the second single-sided area detection box, wherein the preset information area includes at least one of a barcode area, a QR code area, and a text area;

[0042] The predicted probability map is binarized based on the predicted probability map of the preset information region and the preset threshold map to obtain a binarized mask map.

[0043] The preset information region is obtained by finding connected regions on the binarized mask image.

[0044] The image within the preset information area is recognized to obtain the waybill recognition result.

[0045] Optionally, the preset information area includes a barcode area, a QR code area, and a text area;

[0046] The identification unit is used for:

[0047] Decode the barcode area to obtain first decoded information;

[0048] Decode the QR code area to obtain the second decoded information;

[0049] If neither the first decoding information nor the second decoding information includes the waybill number, then the waybill number is obtained by performing waybill recognition on the text area.

[0050] The waybill number is identified as the recognition result.

[0051] Optionally, the first rotation angle is the angle between the detection lower border of the first single-area detection frame and the horizontal axis;

[0052] The correction unit is used for:

[0053] The second rotation angle is determined based on the corresponding border relationship;

[0054] The target rotation angle is determined based on the first rotation angle and the second rotation angle;

[0055] The image within the first single-area detection box is rotated based on the target rotation angle to obtain the second single-area detection box.

[0056] Optionally, the waybill area detection unit is used to obtain a preset waybill area detection model. The preset waybill area detection model is obtained by training the preset detection model based on a preset express image sample set. The preset express image sample set includes multiple express sample images. The express sample images are marked with waybill area annotation boxes and distance values ​​from each pixel in the waybill area annotation box to the four annotation borders of the waybill area annotation box and corresponding annotation labels. The annotation labels are determined based on the correspondence between the waybill area annotation box and each actual border of the actual waybill image.

[0057] The package image is input into the preset shipping label area detection model to obtain the first rotation angle of the first shipping label area detection frame and the distance identification information of the target pixel in the first shipping label area detection frame to each detection border of the first shipping label area detection frame.

[0058] Optionally, the distance identification information is a positive or negative sign of each distance from the target pixel to each detection border of the first single-area detection box.

[0059] Optionally, the acquisition unit is configured to:

[0060] The package to be identified is scanned to determine whether the scan information has been obtained.

[0061] If the barcode information of the package to be identified is not obtained, the package to be identified is photographed to obtain the image of the package.

[0062] Thirdly, this application provides a computer device, the computer device comprising:

[0063] One or more processors;

[0064] Memory; and

[0065] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the express waybill identification method as described in any one of the first aspects.

[0066] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to perform the steps in the express delivery waybill identification method described in any one of the first aspects.

[0067] This application provides a method, apparatus, computer device, and storage medium for recognizing express delivery waybills. The method, after acquiring an image of the express delivery to be recognized, detects the waybill area to obtain a first rotation angle of a first waybill area detection frame and distance information indicating the distance from the target pixel in the first waybill area detection frame to each detection border of the first waybill area detection frame. Based on the distance information, it determines the border correspondence between each detection border of the first waybill area detection frame and each actual border of the actual waybill image. Then, based on the border correspondence and the first rotation angle, it corrects the image within the first waybill area detection frame. Since each detection border corresponds to each actual border, the correction is equivalent to correction based on the actual border and the first rotation angle, which avoids situations such as image inversion after correction, thereby improving the success rate of waybill recognition. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic diagram of a scenario for the express mail waybill recognition system provided in an embodiment of this application;

[0070] Figure 2 This is a schematic flowchart of an embodiment of the express mail waybill recognition method provided in this application;

[0071] Figure 3 This is a schematic diagram of an embodiment of the first single-area detection frame in this application;

[0072] Figure 4 This is a schematic diagram of the border correspondence in the embodiments of this application;

[0073] Figure 5 This is a flowchart illustrating another embodiment of the express mail waybill recognition method provided in this application.

[0074] Figure 6 This is a schematic diagram of an embodiment of the express delivery waybill recognition device provided in this application.

[0075] Figure 7 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0078] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0079] This application provides a method, apparatus, computer device, and storage medium for recognizing express delivery waybills, which will be described in detail below.

[0080] Please see Figure 1 , Figure 1This is a schematic diagram of a scenario for a parcel waybill recognition system provided in an embodiment of this application. The parcel waybill recognition system may include a computer device 100, which integrates a parcel waybill recognition device.

[0081] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0082] In this embodiment, the computer device 100 described above can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device 100 can be a desktop computer, a portable computer, a network server, a handheld computer (Personal Digital Assistant, PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the type of computer device 100.

[0083] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include more than one application scenario. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the document. It is understood that the express waybill identification system may also include one or more other computer devices capable of processing data, which are not specifically limited here.

[0084] In addition, such as Figure 1 As shown, the express delivery waybill recognition system may also include a memory 200 for storing data.

[0085] It should be noted that, Figure 1 The schematic diagram of the express delivery waybill recognition system shown is merely an example. The express delivery waybill recognition system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of express delivery waybill recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0086] First, this application provides a method for recognizing express delivery waybills. The method includes: acquiring an image of an express delivery to be recognized; performing waybill region detection on the express delivery image to obtain a first rotation angle of a first waybill region detection frame and distance identification information of the distance from the target pixel in the first waybill region detection frame to each detection border of the first waybill region detection frame; determining the border correspondence between each detection border of the first waybill region detection frame and each actual border of the actual waybill image based on the distance identification information; correcting the image within the first waybill region detection frame based on the border correspondence and the first rotation angle to obtain a second waybill region detection frame; and recognizing the image within the second waybill region detection frame to obtain a waybill recognition result.

[0087] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the express delivery waybill recognition method in this application, which includes the following steps S201 to S205:

[0088] S201. Obtain the image of the package to be identified.

[0089] In one specific embodiment, a camera or other photographic device is used to take pictures of the packages to be identified on the express delivery system at a preset frequency to obtain images of the packages. The preset frequency can be 1 Hz, 2 Hz, etc., and can be set according to specific circumstances.

[0090] In another specific embodiment, a barcode scanner or similar device is used to scan the package to be identified. The system determines whether barcode information has been obtained. If no barcode information is obtained, it indicates that the package cannot be successfully scanned by the barcode scanner. In this case, a camera or similar device is used to photograph the package, obtaining an image. If barcode information is obtained, the sorting method for the package is determined to be machine sorting. The barcode information can be the package's sorting information, for example, the sorting information indicating the package is destined for compartment A. Barcode scanners have advantages such as fast decoding speed and low cost, but they also have poor environmental adaptability and low decoding success rate in some scenarios. Therefore, first using a barcode scanner to quickly identify packages that can be hardware-decoded, and then using image recognition to identify packages that cannot be hardware-decoded, can improve the success rate of package identification while reducing the cost and increasing the efficiency of package identification.

[0091] S202. Perform label area detection on the express delivery image to obtain the first rotation angle of the first label area detection frame and the distance identification information of the target pixel in the first label area detection frame to each detection border of the first label area detection frame.

[0092] Specifically, a preset waybill region detection model is obtained. This model is trained using a preset express package image sample set. The sample set includes multiple express package images, each labeled with a waybill region bounding box and the distance values ​​from each pixel within the bounding box to the four bounding borders of the bounding box, along with corresponding label identifiers. The label identifiers are determined based on the correspondence between the bounding box and the actual borders of the actual waybill image. The express package image is then input into the preset waybill region detection model to obtain the first rotation angle of the first waybill region detection box and the distance identifier information from the target pixel within the first waybill region detection box to each detection border of the first waybill region detection box.

[0093] In this embodiment, the distance identification information is a positive or negative sign indicating the distance from the target pixel to each detection border of the first single-area detection frame. The first single-area detection frame is a rectangle. The target pixel can be any pixel within the first single-area detection frame.

[0094] See Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the first single-area detection frame in this application.

[0095] Among them, Figure 3 In the diagram, the first label area detection box is a dashed rectangle, and the actual label image is a solid quadrilateral. "Top," "Bottom," "Left," and "Right" correspond to the actual top border, bottom border, left border, and right border of the actual label image, respectively. The target pixel is O, and the distances from the target pixel O to each detection border of the first label area detection box are t, l, r, and b, respectively. Each distance includes a distance value and a corresponding positive or negative sign. For example, if the distances are -10, 9, 8, and -8, then the distance values ​​are 10, 9, 8, and 8, and the positive or negative signs are -, +, +, and -, respectively. That is, the distance identification information is -, +, +, and -. Of course, in other embodiments, the distance identification information can be other identifiers.

[0096] Where t represents the distance from the target pixel to the top border of the first single-region detection box; l represents the distance from the target pixel to the left border of the first single-region detection box; r represents the distance from the target pixel to the right border of the first single-region detection box; and b represents the distance from the target pixel to the bottom border of the first single-region detection box. The first rotation angle α is the angle between the bottom border of the first single-region detection box and the horizontal axis. That is, after the bottom border of the first single-region detection box is rotated clockwise by α, the bottom border of the first single-region detection box is parallel to the horizontal axis.

[0097] In this embodiment, the preset waybill area detection model is obtained by training a preset detection model based on a preset express package image sample set. The preset express package image sample set includes multiple express package sample images. The express package sample images are marked with a waybill area annotation box, the distance value of each pixel in the waybill area annotation box to the four annotation borders of the waybill area annotation box, and the corresponding annotation label. The annotation label is determined according to the correspondence between the waybill area annotation box and each actual border of the actual waybill image.

[0098] It should be noted that the actual borders of the actual shipping label image refer to the borders defined when the actual shipping label image is placed normally. When the actual shipping label image is placed normally, the text on the actual shipping label image is in a straight position.

[0099] Specifically, the preset detection model is the EAST (Efficient and Accuracy Scene Text) model. The EAST model can be decomposed into three parts: feature extraction, feature merging, and output layer. Feature extraction: First, the parameters of a convolutional network pre-trained on the dataset are initialized; then, based on the VGG16 model, four levels of feature maps are extracted from the feature extraction stage, with sizes of 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the input image, respectively. In addition, feature maps after pooling-2 to pooling-5 are extracted for feature merging. Feature merging: Merging is performed layer by layer. In each merging stage, first, the feature map from the previous stage is input into an unpooling layer to increase its size; then, it is merged with the feature map of the current layer; finally, the number of channels and computational cost are reduced by conv1×1; conv3×3 fuses local information to finally produce the output of this merging stage. After the last merging stage, the conv3×3 layer generates the final feature map of the merged branch and sends it to the output layer. Output layer: Contains several conv1×1 operations to project the 32-channel feature map onto a 1-channel fractional feature map and a multi-channel geometric feature map. The geometric output can be either RBOX or QUAD.

[0100] The geometry of the rotated rectangle RBOX output by the EAST model is represented by a 4-channel horizontal bounding box AABB and a 1-channel rotation angle. The four channels of the horizontal bounding box AABB represent the four distances from the pixel location to the top, right, bottom, and left borders of the rotated rectangle RBOX. However, the four sides of the horizontal bounding box AABB output by the EAST model have no actual meaning and cannot accurately correspond to the actual top, bottom, left, and right borders of the label. In other words, the top, right, bottom, and left borders output by the EAST model are not the actual top, right, bottom, and left borders of the actual image.

[0101] In this embodiment of the application, the preset express delivery image sample set includes multiple express delivery sample images. The express delivery sample images are marked with a label area annotation box, the distance value of each pixel in the label area annotation box to the four annotation borders of the label area annotation box, and the corresponding annotation mark.

[0102] In the process of annotating the preset sample set of express delivery images, each annotation border of the label area is mapped to the actual top, bottom, left, and right borders of the actual label image. For example, the sign of the distance from the actual top border of the label area to the pixel is determined to be positive, representing the top border; the sign of the distance from the actual right border of the label area to the pixel is determined to be negative, representing the top border. Therefore, in the prediction stage, by determining the sign of the distances t, l, r, b from the target pixel to the first label area detection box, we can determine the top, bottom, left, and right boundaries of the label corresponding to the rotating rectangle, thus correctly correcting the label and preventing it from being reversed left-right or top-bottom.

[0103] Because the parcel sample images are marked with a bounding box for the shipping label area and the distance values ​​from each pixel within the bounding box to the four borders of the bounding box, along with corresponding label identifiers, the preset shipping label area detection model is trained based on the parcel sample images. Therefore, by inputting the parcel image to be identified into the preset shipping label area detection model, the first rotation angle α of the first shipping label area detection box and the distance identifier information from the target pixel within the first shipping label area detection box to each detection border of the first shipping label area detection box can be obtained. For example, the distance identifier information from the target pixel to each detection border of the first shipping label area detection box is -, +, +, -, respectively.

[0104] S203. Determine the correspondence between each detection border of the first label area detection box and each actual border of the actual label image based on the distance identification information.

[0105] In one specific embodiment, the correspondence between distance identification information and border correspondence can be preset.

[0106] See Table 1 and Figure 4 Table 1 shows the correspondence between distance identification information and border information. Figure 4 This is a schematic diagram of the border correspondence in the embodiments of this application.

[0107] Combined with Table 1, Figure 3 as well as Figure 4Here, "top" and "left" represent the actual positions of the borders; the distances from the target pixel to each detection border of the first single-area detection box are t, l, r, and b, respectively. The correspondence between the distance identifier information and the border correspondence is as follows:

[0108] If the distance identification information is the first preset distance identification information: +, -, +, -; and the border correspondence is the first border correspondence (a): the detected upper border corresponds to the actual left border, the detected right border corresponds to the actual upper border, the detected lower border corresponds to the actual right border, and the detected left border corresponds to the actual lower border. At this time, rotating the first label area detection box 90 degrees clockwise will make it the same as the actual label image in terms of orientation.

[0109] If the distance identifier information is the second preset distance identifier information: +, +, -, -; and the border correspondence is the second border correspondence (b): the detected upper border corresponds to the actual upper border, the detected right border corresponds to the actual right border, the detected lower border corresponds to the actual lower border, and the detected left border corresponds to the actual left border. In this case, the first label area detection box is in the same orientation as the actual label image.

[0110] If the distance identification information is the third preset distance identification information: -, -, +, +; and the border correspondence is the third border correspondence (c): the detected top border corresponds to the actual bottom border, the detected right border corresponds to the actual left border, the detected bottom border corresponds to the actual top border, and the detected left border corresponds to the actual right border, then rotating the first label area detection box 180 degrees clockwise will make it the same as the actual label image in terms of orientation.

[0111] If the distance identifier information is the fourth preset distance identifier information: -, +, -, +; and the border correspondence is the fourth border correspondence (d): the detected top border corresponds to the actual right border, the detected right border corresponds to the actual bottom border, the detected bottom border corresponds to the actual left border, and the detected left border corresponds to the actual top border. At this time, rotating the first label area detection box 270 degrees clockwise will make it the same as the actual label image in terms of orientation.

[0112] t l r b First preset distance marker information (a) + - + - First preset distance marker information (b) + + - - First preset distance marker information (c) - - + + First preset distance marker information (d) - + - +

[0113] Table 1: Correspondence between distance marker information and border information

[0114] S204. Based on the correspondence of the borders and the first rotation angle, the image within the first single-region detection box is corrected to obtain the second single-region detection box.

[0115] In this embodiment of the application, the image within the first single-region detection box is corrected based on the border correspondence and the first rotation angle to obtain the second single-region detection box, including:

[0116] (1) Determine the second rotation angle based on the correspondence of the borders.

[0117] Specifically, if the border correspondence is the first border correspondence (a), the first label area detection box is rotated 90 degrees clockwise to match the actual label image's orientation, and the second rotation angle is 90 degrees. If the border correspondence is the second border correspondence (b), the first label area detection box is positioned the same as the actual label image, and the second rotation angle is 0 degrees. If the border correspondence is the third border correspondence (c), the first label area detection box is rotated 180 degrees clockwise to match the actual label image's orientation, and the second rotation angle is 180 degrees. If the border correspondence is the fourth border correspondence (d), the first label area detection box is rotated 270 degrees clockwise to match the actual label image's orientation, and the second rotation angle is 270 degrees.

[0118] (2) Determine the target rotation angle based on the first rotation angle and the second rotation angle.

[0119] The sum of the first rotation angle and the second rotation angle is determined as the target rotation angle. For example, if the distance identification information is the first preset distance identification information: +, -, +, -; and the border correspondence is the first border correspondence (a), the target rotation angle is (α+90) degrees.

[0120] (3) Rotate the image within the first single-region detection box based on the target rotation angle to obtain the second single-region detection box.

[0121] Specifically, the image within the first single-region detection box is rotated counterclockwise by (α+90) degrees to obtain the second single-region detection box. At this point, the position of the second single-region detection box is corrected.

[0122] S205. Recognize the image within the second label area detection box to obtain the label recognition result.

[0123] In this embodiment of the application, recognizing the image within the second label region detection box to obtain the label recognition result may include:

[0124] (1) Input the image within the second single-region detection box into the preset information region recognition model to obtain the predicted probability map of the preset information region within the second single-region detection box, wherein the preset information region includes at least one of the barcode region, the QR code region, and the text region.

[0125] A barcode is a graphic identifier that uses multiple black bars and spaces of varying widths, arranged according to specific encoding rules, to represent a set of information. Common barcodes consist of parallel lines of black and white bars with significantly different reflectivities. The barcode area is the same area where the barcode on a shipping label is located.

[0126] A two-dimensional barcode, also known as a QR code, is a popular encoding method on mobile devices in recent years. A QR code is a black-and-white graphic that uses specific geometric shapes arranged according to a certain pattern on a two-dimensional plane to record data symbols. It cleverly utilizes the concept of "0" and "1" bit streams, the foundation of computer logic, using several geometric shapes corresponding to binary to represent textual and numerical information. This information is automatically processed by image input devices or photoelectric scanning devices. It shares some common characteristics with barcode technology: each code system has its specific character set; each character occupies a certain width; and it has certain verification functions. It also has the ability to automatically identify information in different rows and handle graphic rotation changes. The QR code area is the area where the two-dimensional barcode is located.

[0127] The text region is the area containing the waybill number. The predicted probability map represents the probability that each pixel in the image belongs to the text. The predicted probability map of the preset information region represents the probability that each pixel in the second waybill region detection box belongs to the preset information.

[0128] When the preset information area includes a barcode area, a QR code area, and a text area, the predicted probability map of the preset information area includes a barcode area predicted probability map, a QR code area predicted probability map, and a text area predicted probability map. The barcode area predicted probability map represents the probability that each pixel belongs to the barcode, the QR code area predicted probability map represents the probability that each pixel belongs to the QR code, and the text area predicted probability map represents the probability that each pixel belongs to the text.

[0129] The preset information region recognition model is an improvement on DBNet, a text detection scheme based on image segmentation. DBNet's prediction layer (head) uses a binary classification approach, outputting a single-layer prediction probability map that determines the probability of each pixel belonging to text, with a value ranging from 0 to 1. The preset information region recognition model's prediction layer uses a multi-class classification approach, outputting a three-layer prediction probability map that identifies barcode regions, QR code regions, and text regions.

[0130] Of course, corresponding preset information region recognition models can also be trained separately for the barcode region, QR code region, and text region. The barcode region in the second single-page area detection box can be recognized based on the preset information region recognition model corresponding to the barcode region; the QR code region in the second single-page area detection box can be recognized based on the preset information region recognition model corresponding to the QR code region; and the text region in the second single-page area detection box can be recognized based on the preset information region recognition model corresponding to the text region.

[0131] (2) Binarize the prediction probability map according to the prediction probability map of the preset information region and the preset threshold map to obtain the binarized mask map.

[0132] Specifically, a backbone network such as ResNet18 or ResNet50 is used to extract features from a preset information region, resulting in a feature map F. A prediction probability map is then obtained based on feature map F. Finally, the prediction probability map is binarized using the prediction probability map and a preset threshold map to obtain a binarized mask. Standard binarization (SB) or differentiable binarization (DB) can be used for binarization.

[0133] The preset threshold map represents the confidence threshold of the target boundary. The original DBNet only has one layer, and we have also extended the preset threshold map to three layers, including the barcode threshold map, the QR code threshold map, and the text threshold map.

[0134] (3) Find the connected regions on the binary mask to obtain the preset information region.

[0135] (4) Recognize the image within the preset information area to obtain the waybill recognition result.

[0136] Specifically, the barcode area, QR code area, and text area in the preset information area are cropped out separately, and the barcode area, QR code area, and text area are identified separately to obtain the waybill recognition result.

[0137] In one specific embodiment, the preset information area includes a barcode area, a QR code area, and a text area. Recognition is performed on the image within the preset information area to obtain the waybill recognition result, including:

[0138] Decode the barcode area to obtain the first decoding information; decode the QR code area to obtain the second decoding information; wherein, mature decoding tools such as zbar or zxing can be used to decode the barcode area and the QR code area.

[0139] If neither the first nor the second decoding information includes the waybill number, then waybill recognition is performed on the text area to obtain the waybill number; the waybill number is then determined as the recognition result. Specifically, optical character recognition (OCR) is performed on the text area to obtain the waybill number. OCR refers to the process by which electronic devices (such as scanners or digital cameras) examine characters printed on paper, determine their shape by detecting dark and light patterns, and then translate the shape into computer text using character recognition methods; that is, for printed characters, optical methods are used to convert the text in paper documents into black and white dot matrix image files, and recognition software converts the text in the image into text format for further editing and processing by word processing software.

[0140] If both the first and second decoded information include the waybill number, then it is determined whether the waybill numbers in the first and second decoded information match. If they match, the waybill number in the first and second decoded information is determined as the recognition result. If the first and second decoded information do not match, the waybill number is recognized in the text area; the waybill number is then determined as the recognition result.

[0141] Furthermore, if the waybill recognition result contains the waybill number, the sorting method for the parcel to be identified is determined to be machine sorting; if the waybill recognition result does not contain the waybill number, the sorting method for the parcel to be identified is determined to be manual sorting.

[0142] Furthermore, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating another embodiment of the express delivery waybill recognition method provided in this application. The express delivery waybill recognition method includes the following steps S301 to S309:

[0143] S301. Scan the package to be identified and determine whether the scan information has been obtained.

[0144] Specifically, the package to be identified is scanned to determine whether the scan information is obtained. If the scan information is obtained, then S308 is executed: the sorting method of the package to be identified is determined to be machine sorting; if the scan information is not obtained, then S302 is executed: the package to be identified is photographed to obtain the package image.

[0145] S302. Take a picture of the package to be identified to obtain an image of the package.

[0146] In this embodiment, the specific steps of S302 can be found in S201, and will not be repeated here.

[0147] S303. Perform label area detection on the express delivery image to obtain the first rotation angle of the first label area detection frame and the distance identification information of the target pixel point in the first label area detection frame to each detection border of the first label area detection frame.

[0148] In this embodiment, the specific steps of S303 can be found in S202, and will not be repeated here.

[0149] S304. Determine the correspondence between each detection border of the first label area detection box and each actual border of the actual label image based on the distance identification information.

[0150] In this embodiment, the specific steps of S304 can be found in S203, and will not be repeated here.

[0151] S305. Based on the correspondence of the borders and the first rotation angle, the image within the first single-region detection box is corrected to obtain the second single-region detection box.

[0152] In this embodiment, the specific steps of S305 can be found in S204, and will not be repeated here.

[0153] S306. Recognize the image within the second label area detection box to obtain the label recognition result.

[0154] In this embodiment, the specific steps of S306 can be found in S205, and will not be repeated here.

[0155] S307. Determine whether the waybill recognition result contains the waybill number.

[0156] Specifically, if the waybill recognition result contains the waybill number, then execute S308: determine the sorting method of the parcel to be identified as machine sorting; if the waybill recognition result does not contain the waybill number, then execute S309: determine the sorting method of the parcel to be identified as manual sorting.

[0157] S308. The sorting method for the parcels to be identified is determined to be machine sorting.

[0158] S309. The sorting method for the parcels to be identified is determined to be manual sorting.

[0159] To better implement the express delivery waybill recognition method in the embodiments of this application, based on the express delivery waybill recognition method, the embodiments of this application also provide an express delivery waybill recognition device, such as... Figure 6 As shown, the express delivery waybill recognition device 400 includes:

[0160] The acquisition unit 401 is used to acquire an image of the express package to be identified;

[0161] The waybill area detection unit 402 is used to perform waybill area detection on the express image and obtain the first rotation angle of the first waybill area detection frame and the distance identification information of the target pixel point in the first waybill area detection frame to each detection border of the first waybill area detection frame.

[0162] The determining unit 403 is used to determine the correspondence between each detection border of the first label area detection box and each actual border of the actual label image based on the distance identification information.

[0163] The correction unit 404 is used to correct the image within the first single-area detection box based on the bounding box correspondence and the first rotation angle to obtain the second single-area detection box.

[0164] The recognition unit 405 is used to recognize the image within the second label area detection box to obtain the label recognition result.

[0165] Optionally, the identification unit 405 is used for:

[0166] The image within the second single-region detection box is input into the preset information region recognition model to obtain the predicted probability map of the preset information region within the second single-region detection box. The preset information region includes at least one of the following: barcode region, QR code region, and text region.

[0167] The predicted probability map is binarized based on the predicted probability map of the preset information region and the preset threshold map to obtain a binarized mask map.

[0168] Find connected regions on the binary mask to obtain the preset information region;

[0169] The image within the preset information area is recognized to obtain the waybill recognition result.

[0170] Optionally, the preset information area includes a barcode area, a QR code area, and a text area;

[0171] The identification unit 405 is used for:

[0172] Decode the barcode area; obtain the first decoded information;

[0173] Decode the QR code area to obtain the second decoded information;

[0174] If neither the first nor the second decoding information includes the waybill number, then the waybill number is obtained by performing waybill recognition on the text area.

[0175] The waybill number is used as the identification result.

[0176] Optionally, the first rotation angle is the angle between the detection lower border of the first single-area detection frame and the horizontal axis;

[0177] Correction unit 404 is used for:

[0178] The second rotation angle is determined based on the correspondence of the borders;

[0179] The target rotation angle is determined based on the first rotation angle and the second rotation angle.

[0180] The image within the first single-region detection box is rotated based on the target rotation angle to obtain the second single-region detection box.

[0181] Optionally, the waybill area detection unit 402 is used to obtain a preset waybill area detection model. The preset waybill area detection model is obtained by training the preset detection model based on a preset express image sample set. The preset express image sample set includes multiple express sample images. The express sample images are marked with waybill area annotation boxes and distance values ​​from each pixel in the waybill area annotation box to the four annotation borders of the waybill area annotation box and corresponding annotation labels. The annotation labels are determined according to the correspondence between the waybill area annotation box and each actual border of the actual waybill image.

[0182] The image of the express package is input into the preset waybill area detection model to obtain the first rotation angle of the first waybill area detection frame and the distance identification information of the target pixel in the first waybill area detection frame to each detection border of the first waybill area detection frame.

[0183] Optionally, the distance identification information is a positive or negative sign of each distance from the target pixel to each detection border of the first single-area detection box.

[0184] Optionally, the acquisition unit 401 is used for:

[0185] The system scans the code on the package to be identified and determines whether it has obtained the scan information.

[0186] If the barcode information of the package to be identified is not obtained, the package will be photographed to obtain an image of the package.

[0187] This application also provides a computer device that integrates any of the express delivery waybill recognition devices provided in this application. The computer device includes:

[0188] One or more processors;

[0189] Memory; and

[0190] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor in the steps of the express waybill identification method in any of the embodiments described above.

[0191] like Figure 7 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0192] The computer device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will understand that... Figure 5 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0193] Processor 501 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory 502, and by calling data stored in memory 502, thereby providing overall monitoring of the computer device. Optionally, processor 501 may include one or more processing cores; processor 501 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 501.

[0194] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0195] The computer equipment also includes a power supply 503 that supplies power to the various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0196] The computer device may also include an input unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0197] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 501 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application programs stored in the memory 502 to realize various functions, as follows:

[0198] Acquire an image of the package to be identified; perform label area detection on the package image to obtain a first rotation angle of the first label area detection frame and distance identification information of the distance from the target pixel in the first label area detection frame to each detection border of the first label area detection frame; determine the border correspondence between each detection border of the first label area detection frame and each actual border of the actual label image based on the distance identification information; correct the image within the first label area detection frame based on the border correspondence and the first rotation angle to obtain a second label area detection frame; identify the image within the second label area detection frame to obtain the label identification result.

[0199] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0200] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the express delivery waybill identification methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0201] Acquire an image of the package to be identified; perform label area detection on the package image to obtain a first rotation angle of the first label area detection frame and distance identification information of the distance from the target pixel in the first label area detection frame to each detection border of the first label area detection frame; determine the border correspondence between each detection border of the first label area detection frame and each actual border of the actual label image based on the distance identification information; correct the image within the first label area detection frame based on the border correspondence and the first rotation angle to obtain a second label area detection frame; identify the image within the second label area detection frame to obtain the label identification result.

[0202] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0203] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0204] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0205] The foregoing has provided a detailed description of a method, apparatus, computer device, and storage medium for identifying express delivery waybills according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying a courier label, characterized by, The express delivery waybill recognition method includes: Obtain an image of the package to be identified; The package image is subjected to label area detection to obtain a first rotation angle of the first label area detection frame and distance identification information of the distance from the target pixel in the first label area detection frame to each detection border of the first label area detection frame; the distance identification information is a positive or negative sign of each distance from the target pixel to each detection border of the first label area detection frame; the first rotation angle is the angle between the detection bottom border of the first label area detection frame and the horizontal axis. The correspondence between each detection border of the first label area detection box and each actual border of the actual label image is determined based on the distance identification information. The image within the first single-area detection box is corrected based on the border correspondence and the first rotation angle, wherein a second rotation angle is determined based on the border correspondence; a target rotation angle is determined based on the first rotation angle and the second rotation angle; and the image within the first single-area detection box is rotated based on the target rotation angle to obtain a second single-area detection box. The image within the second label area detection box is recognized to obtain the label recognition result.

2. The express delivery waybill identification method according to claim 1, characterized in that, The step of recognizing the image within the second label region detection box to obtain the label recognition result includes: The image within the second single-sided area detection box is input into the preset information area recognition model to obtain the predicted probability map of the preset information area within the second single-sided area detection box, wherein the preset information area includes at least one of a barcode area, a QR code area, and a text area; The predicted probability map is binarized based on the predicted probability map of the preset information region and the preset threshold map to obtain a binarized mask map. The preset information region is obtained by finding connected regions on the binarized mask image. The image within the preset information area is recognized to obtain the waybill recognition result.

3. The express delivery waybill identification method according to claim 2, characterized in that, The preset information area includes a barcode area, a QR code area, and a text area; The step of recognizing the image within the preset information area to obtain the waybill recognition result includes: Decode the barcode area to obtain first decoded information; Decode the QR code area to obtain the second decoded information; If neither the first decoding information nor the second decoding information includes the waybill number, then the waybill number is obtained by performing waybill recognition on the text area. The waybill number is identified as the recognition result.

4. The express delivery waybill identification method according to any one of claims 1 to 3, characterized in that, The step of performing label region detection on the express delivery image to obtain the first rotation angle of the first label region detection frame and the distance identification information of the distance from the target pixel in the first label region detection frame to each detection border of the first label region detection frame includes: A preset waybill area detection model is obtained. The preset waybill area detection model is trained on a preset express delivery image sample set. The preset express delivery image sample set includes multiple express delivery sample images. The express delivery sample images are marked with waybill area annotation boxes and distance values ​​from each pixel in the waybill area annotation box to the four annotation borders of the waybill area annotation box and corresponding annotation labels. The annotation labels are determined according to the correspondence between the waybill area annotation box and each actual border of the actual waybill image. The package image is input into the preset shipping label area detection model to obtain the first rotation angle of the first shipping label area detection frame and the distance identification information of the target pixel in the first shipping label area detection frame to each detection border of the first shipping label area detection frame.

5. The express delivery waybill identification method according to any one of claims 1 to 3, characterized in that, The process of acquiring the image of the package to be identified includes: The package to be identified is scanned to determine whether the scan information has been obtained. If the barcode information of the package to be identified is not obtained, the package to be identified is photographed to obtain the image of the package.

6. A parcel waybill recognition device, characterized in that, The express delivery waybill identification device, using the express delivery waybill identification method as described in any one of claims 1 to 5, comprises: The acquisition unit is used to acquire images of the express package to be identified. A waybill area detection unit is used to perform waybill area detection on the express image to obtain a first rotation angle of a first waybill area detection frame and distance identification information of the distance from the target pixel in the first waybill area detection frame to each detection border of the first waybill area detection frame; the distance identification information is a positive or negative sign of each distance from the target pixel to each detection border of the first waybill area detection frame; the first rotation angle is the angle between the detection bottom border of the first waybill area detection frame and the horizontal axis. The determining unit is used to determine the correspondence between each detection border of the first label area detection box and each actual border of the actual label image based on the distance identification information. A correction unit is used to correct the image within the first single-area detection box based on the border correspondence and the first rotation angle, wherein a second rotation angle is determined based on the border correspondence; a target rotation angle is determined based on the first rotation angle and the second rotation angle; and the image within the first single-area detection box is rotated based on the target rotation angle to obtain a second single-area detection box. The recognition unit is used to recognize the image within the second label area detection box to obtain the label recognition result.

7. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the express waybill identification method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps in the express waybill identification method according to any one of claims 1 to 5.

Images