A package quantity detection method and device and a storage medium

Abstract

The present application relates to a kind of parcel quantity detection method, device and storage medium, wherein the detection method includes: obtaining a group of parcel on-site image and on-site weight;According to the parcel identification number in the on-site image, the corresponding pre-stored weight is searched;If the on-site weight and the pre-stored weight do not match, the on-site image is input into target detection model to carry out target prediction, and the boundary box information of the parcel in the on-site image is obtained;According to the boundary box information, the quantity of parcel is identified.The present application can realize the quantity detection of most stacked parcels, so as to help determine the accurate weight of parcel.The present application also has the advantages of high detection efficiency and precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of logistics transportation, in particular to a parcel quantity detection method, device and storage medium. BACKGROUND

[0002] In the logistics transportation industry, the weight of a parcel is one of the main bases for collecting transportation fees, and an inaccurate weight will cause a loss of fees for a logistics company or a customer. Therefore, how to find the weight error of a parcel becomes an important link for a logistics company to recover transportation fees.

[0003] Taking the express industry as an example, an express courier needs to weigh a parcel before collecting the parcel from a customer, and upload the waybill number together with the weighed weight to a background management system; when the parcel reaches an automatic sorting line of an express company, the sorting line automatically re-weighs the parcel to determine the weight.

[0004] However, the placement of the parcel on the sorting line is usually not standardized, for example, two or more parcels may be stacked together, which can easily cause the weight weighed on the sorting line to be wrong. For example, a group of parcels weighed at present is stacked by two parcels, and only the upper one is automatically identified, while the weight obtained by weighing is the overall weight of the two parcels, which will only be regarded as the weight of the upper one, which will cause the weight weighed on the sorting line to be obviously different from the weight uploaded by the express courier. Once the express company recovers the transportation fees from the customer according to the weight weighed on the sorting line, it will cause the customer to have unnecessary expenses or lead to disputes and other adverse consequences.

[0005] Therefore, how to automatically detect the number of stacked parcels becomes a technical problem to be solved. SUMMARY

[0006] The purpose of the present application is to disclose a parcel quantity detection method to realize quantity detection of most stacked parcels.

[0007] The present application discloses a parcel quantity detection method, comprising the following steps:

[0008] A1: obtaining an on-site image and an on-site weight of a group of parcels;

[0009] A2: searching for a corresponding pre-stored weight according to a parcel identification number in the on-site image;

[0010] A3: if the on-site weight and the pre-stored weight do not match, inputting the on-site image into a target detection model to obtain boundary box information of the parcels in the on-site image;

[0011] A4: identifying the number of parcels according to the boundary box information.

[0012] The present application firstly selects a package whose field weight does not match the pre-stored weight, inputs the field image of the package into a target detection model to perform target prediction, and obtains the bounding box information of the package in the field image, so that the number of the package can be identified according to the bounding box information, the detection of the number of the package is realized, and the efficiency and accuracy of the detection are improved.

[0013] As preferred, after step A4, further comprising:

[0014] A5: if the number of the package is identified as one according to the bounding box information, the field image is cropped based on the bounding box information to obtain a package image;

[0015] A6: the package image is input into a classification detection model to perform classification prediction, and a classification result of the number of the package is obtained as one or more.

[0016] As preferred, the bounding box information includes a result vector; the result vector includes coordinate values, confidence and type values related to the package.

[0017] As preferred, the package identification number is a symbol, number or identification information that can distinguish the package from other packages.

[0018] As preferred, the field image is an image automatically collected on a package sorting line for a group of packages currently weighed, and the field weight is the weight of the group of packages weighed.

[0019] As preferred, the target detection model is used to perform target prediction in step A3 after being trained; the training of the target detection model includes the following steps:

[0020] B1: label the bounding box information of the package in the field image for training;

[0021] B2: export the bounding box information into a label file in a format suitable for the training framework adopted;

[0022] B3: convert the label file into a standard file and add it to the training set;

[0023] train based on the training framework, the target detection algorithm and the training set.

[0024] As preferred, before step B1, further comprising:

[0025] B0: perform target prediction on the field image for training by the target detection model to obtain the bounding box information of the package in the field image for training.

[0026] As preferred, the classification detection model is used to perform the classification prediction in step A6 after training; the training of the classification detection model comprises the following steps:

[0027] C1: the on-site image output after performing the target prediction on the target detection model is cropped according to the bounding box of the package contained therein to obtain a training package image;

[0028] C2: the training package image is stored in one of the two image sets of single piece and multiple pieces according to the number of packages contained therein;

[0029] C3: the two image sets are trained by using a deep learning classification algorithm.

[0030] The application also discloses a package quantity detection device comprising the following functional modules:

[0031] An image acquisition module is used to acquire on-site images of a group of packages;

[0032] A weight acquisition module is used to acquire on-site weights of the group of packages;

[0033] A search module is used to match the corresponding pre-stored weights according to the package identification numbers in the on-site images;

[0034] A target prediction module is used to input the on-site images into a target detection model for target prediction when the on-site weights do not match the pre-stored weights, to obtain the bounding box information of the packages in the on-site images; and the number of packages is identified according to the bounding box information.

[0035] As preferred, the package quantity detection device further comprises:

[0036] A cropping module is used to crop the on-site images based on the bounding box information when the number of packages identified by the bounding box information is single, to obtain a package image;

[0037] A classification prediction module is used to input the package image into a classification detection model for classification prediction, to correspondingly obtain a quantity classification result of the package as single piece or multiple pieces.

[0038] The application also discloses a computer readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to perform the steps of the package quantity detection method of any one of the above schemes.

[0039] Beneficial effects: the application can realize quantity detection of most stacked packages, thereby helping to determine the accurate weight of the packages and helping the logistics company or the customer to reduce the freight loss. Meanwhile, the application also has the advantages of high detection efficiency and precision. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0041] Figure 1 is a flowchart of the package quantity detection method disclosed in embodiment one.

[0042] Figure 2 is a flowchart of the package quantity detection method disclosed in embodiment two.

[0043] Figure 3 is a flowchart of the training of the target detection model in the package quantity detection method disclosed in embodiment three.

[0044] Figure 4 is a flowchart of the training of the classification detection model in the package quantity detection method disclosed in embodiment four.

[0045] Figure 5 is a structural block diagram of the package quantity detection device disclosed in embodiment five. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments only constitute some embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the scope of the present application.

[0047] Embodiment one

[0048] Taking a courier company as an example, a courier company with more advanced operation technology is configured with an automatic sorting line, which can sort packages according to their routing information (destination). The automatic sorting line is provided with a weighing device and a shooting device, which are respectively used to collect the on-site weight and on-site image of the package when it is weighed.

[0049] In this embodiment, the on-site weight refers to the weight collected on the site of the automatic sorting line for a group of packages; the on-site image refers to the image collected on the site of the automatic sorting line; since the image is collected at the same time, the background will be collected inevitably, so the on-site image will include not only the package itself, but also a part of the site background where the package is located.

[0050] Please refer to Figure 1, the following will be introduced in detail a package quantity detection method disclosed in embodiment one, specifically, mainly includes the following steps S10-S40:

[0051] S10: Obtain the on-site image and on-site weight of a group of packages.

[0052] In an ideal state, the number of a group of packages weighed (i.e. weighed by a weighing device) is only one, i.e. single piece; and when stacking occurs, the number of a group of packages is two or more, i.e. multiple pieces. Therefore, the number of packages contained in "a group of packages" in step S10 can be single piece or multiple pieces, depending on the actual situation.

[0053] For understanding reference, the on-site image in step S10 can refer to an image automatically collected on a package sorting line for the current group of packages weighed, and the on-site weight can refer to the weight obtained by weighing the group of packages.

[0054] S20: Search for the corresponding pre-stored weight according to the package identification number in the on-site image.

[0055] Since the express industry generally uses waybill number as the identification number of the package, the package identification number in this embodiment refers to the waybill number. In other embodiments, the identification number of the package can refer to any symbol, number or other identifying information that can distinguish the package from other packages.

[0056] When collecting the package of the customer, the express delivery personnel usually needs to weigh the package, and the weight is one of the basis for collecting the delivery fee, and the weight also needs to be uploaded to the background management system of the express company together with the package identification number (waybill number). Therefore, the pre-stored weight in this embodiment refers to the weight data uploaded to the background management system by the express delivery personnel after the aforementioned weighing. In other embodiments, it can refer to weight data pre-stored by any other suitable operation. Step S20 can search for the corresponding pre-stored weight through the aforementioned package identification number.

[0057] S30: If the on-site weight and the pre-stored weight do not match, input the on-site image into a target detection model for target prediction to obtain the bounding box information of the package in the on-site image.

[0058] Generally, the on-site weight and the pre-stored weight are not matched only when the packages are stacked (if the error between the two is within a certain range, it can be considered to be matched), and generally the on-site weight is greater than the pre-stored weight. Therefore, once the on-site weight is greater than the pre-stored weight, it can be determined to a certain extent that the packages are stacked, that is, the weighed package is not a single piece but multiple pieces. However, in order to obtain a more objective and accurate judgment result, step S30 inputs the on-site image into the target detection model for target prediction to obtain the bounding box information of the package in the on-site image as the recognition basis. For example, only one set of bounding box information identifies that the number of targets (packages) is a single piece, and if there are multiple sets of bounding box information, the number of targets (packages) is identified as multiple pieces.

[0059] For reference, the target detection model can use a target detection algorithm such as YOLOv3 algorithm, RetinaNet algorithm, etc. for target prediction.

[0060] S40: Identify the number of packages according to the bounding box information.

[0061] The bounding box information of the package includes information in any format that can be used to describe the position of the package in the on-site image. The present embodiment can output a result vector such as [x1, y1, x2, y2, score, label] through the target detection model; wherein [x1, y1, x2, y2] are the coordinate values of the four vertices of the bounding box (rectangular box) of the package, score is the confidence, and label is the type value (the target prediction of the present embodiment has only one type, i.e. package). According to the number of result vectors, the number of targets can be identified, and the type of the target (i.e. package) and the confidence of the prediction result can be specifically identified.

[0062] In actual application, due to the different light brightness on the sorting line, the images collected in the dark light condition may cause missed detection, especially for some black packages. Therefore, the on-site image input into the target detection model can also be preprocessed by using a limited contrast adaptive histogram equalization algorithm, including: dividing the on-site image into blocks, calculating the histogram first, then pruning the histogram, and equalizing; by traversing and operating each image block, linear interpolation is performed between the blocks; and performing a layer color mixing operation with the original image. Through this preprocessing, the brightness of the relatively dark picture is improved, and the edge is well maintained, reducing the missed detection problem of packages in a dark environment.

[0063] Through the above analysis, the embodiment first selects a package whose on-site weight does not match the pre-stored weight, inputs the on-site image of the package into the target detection model to perform target prediction, and obtains the bounding box information of the package in the on-site image, so that the number of the package can be identified according to the bounding box information, the detection of the number of the package is realized, and the efficiency and accuracy of the detection are improved.

[0064] Embodiment two

[0065] Embodiment one can realize the detection of the number of stacked packages in most cases, but there is still room for further improvement of the detection capability for the case of more serious stacking. Based on this, embodiment two discloses a package number detection method with more superior detection capability.

[0066] Please refer to Figure 2 , the package number detection method disclosed by embodiment two includes steps S10 to S40 of embodiment one, and the improvement is that steps S50 to S60 are further included after step S40:

[0067] S50: If the number of the package is identified as one according to the bounding box information, the on-site image is cropped based on the bounding box information to obtain a package image.

[0068] For the result identified by the foregoing step S40, if the number of the package is identified as one, it is considered that there may be a case that multiple (more seriously stacked packages) are incorrectly identified as one. Therefore, step S50 mainly processes the case that the number of the package is identified as one.

[0069] Specifically, the target detection model can generate a bounding box around the package in the on-site image according to the bounding box information, and step S50 crops according to the bounding box, removes the background part outside the bounding box, and retains the package part inside the bounding box. The retained package part is referred to as a package image in this embodiment (since some background information is helpful as a prediction reference, the actual retained package image can be slightly larger than the area of the bounding box).

[0070] S60: Input the package image into a classification detection model to perform classification prediction, and correspondingly obtain a package number classification result as one or multiple.

[0071] Since step S60 mainly predicts one of the two results, i.e., one or multiple, the classification detection model can output probability values of the two classification results, for example, the probability of one is 0.7 and the probability of multiple is 0.3, which indicates that the classification prediction result is more inclined to consider that the number of the package is one.

[0072] As a reference, the classification detection model can adopt a classification algorithm such as EfficientNet-b3 algorithm of deep learning for classification prediction.

[0073] Since Example Two is a further improvement based on Example One, its specific implementation can be understood in combination with Example One and has the beneficial effects of Example One, which will not be repeated here.

[0074] Example Three

[0075] It can be understood that in Example One and Example Two, the target detection model is by default a mature model that has been trained. And for the training of the target detection model, there are mature solutions in the prior art. However, Example Three further discloses a training scheme more suitable for the target detection model involved in the present application.

[0076] Example Three discloses a package quantity detection method, which includes any scheme of Example One or Example Two, that is, only the training method of the target detection model is improved based on Example One or Example Two.

[0077] Please refer to Figure 3 In the present embodiment, the target detection model is trained by the following steps L11-L14 and used to perform target prediction in step S30:

[0078] L11: Label the bounding box information of the package in the on-site image for training.

[0079] Specifically, before the target detection model is formally put into use, a part of images collected from the on-site of the automatic sorting line can be used as on-site images for training. These on-site images can be imported into some labeling tools (such as Colabeler) for labeling, and the bounding box of the package and the bounding box information can be labeled.

[0080] L12: Export the bounding box information into a labeling file in a format suitable for the training framework adopted.

[0081] For example, by adopting the training framework of MMDetection and the COCO format training set, the bounding box information can be exported into a json format labeling file, and each image corresponds to a json format labeling file.

[0082] L13: Convert the labeling file into a standard file and add it to the training set.

[0083] For example, the json format labeling file is converted into a standard file in COCO format and added to the training set.

[0084] L14: training the target detection algorithm based on the training framework and the training set.

[0085] For example, the training framework based on MMDetection trains the YOLOv3 algorithm and the COCO format training set.

[0086] There is usually an evaluation standard for whether the target detection model is optimal. For example, a mechanism called mean average precision (mAP) can be used as an evaluation standard. The higher the mean average precision, the better the model effect.

[0087] The embodiment can also use a more efficient labeling scheme. Specifically, as a preferred embodiment, step L11 is preceded by:

[0088] L10: obtaining the bounding box information of the parcel in the training field image by target prediction of the training field image by the target detection model.

[0089] Specifically, the training field image can be imported into the target detection model to perform preliminary target prediction and obtain the bounding box information of the parcel. The bounding box information is imported into the labeling tool to form pre-labeled information, thereby saving a part of the labeling of the field image and improving the efficiency.

[0090] Embodiment Four

[0091] Similarly, in Embodiment Two and Embodiment Three, the classification detection model is by default a mature model. And for the training of the classification detection model, there are mature schemes in the prior art. The purpose of Embodiment Four is to disclose a training scheme more suitable for the classification detection model involved in the present application.

[0092] Embodiment Four discloses a parcel quantity detection method, which includes any scheme of Embodiment Two or Embodiment Three, that is, only the training method of the classification detection model is improved based on Embodiment Two or Embodiment Three.

[0093] Please refer to Figure 4 In the embodiment, the classification detection model is trained by the following steps M11-M13 and used to perform classification prediction in step S60:

[0094] M11: cropping the field image output after target prediction of the target detection model according to the bounding box of the parcel therein to obtain the training parcel image.

[0095] Since some background information helps as a prediction reference, the actual retained package image after cropping in step M11 can be slightly larger than the area of the bounding box. For example, a package image equivalent to 1.2 times the area of the bounding box can be retained.

[0096] M12: According to the number of packages, the training package image is stored in one of the two image sets of single-piece and multi-piece.

[0097] Specifically, two folders can be established, one as a single-piece image set and the other as a multi-piece image set. Through manual viewing, training package images of single-piece packages are stored in the single-piece image set, and training package images of multi-piece packages are stored in the multi-piece image set.

[0098] M13: Training using the two image sets by a deep learning classification algorithm.

[0099] Specifically, step M13 can use a deep learning classification algorithm such as EfficientNet-b3 algorithm for training.

[0100] Based on the above detailed introduction of package quantity detection, it can be understood that the above package detection method can be embodied by computer-related components. For example, a computer readable storage medium has a computer program stored thereon, which is loaded by a processor to execute the steps in the package quantity detection method of any of the above embodiments. The computer readable storage medium includes but is not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0101] Embodiment five

[0102] Please refer to Figure 5 Based on the above embodiments one to four, this embodiment discloses a package quantity detection device, which includes the following functional modules:

[0103] Image acquisition module 10: for acquiring live images of a group of packages. The image acquisition module 10 can be a shooting device such as an industrial camera.

[0104] Weight acquisition module 20: for acquiring live weights of the group of packages. The weight acquisition module 20 can be a weighing device such as a dynamic scale that can automatically weigh.

[0105] The search module 30 is configured to match the package identification number in the on-site image to the corresponding pre-stored weight. The search module 30 can be a functional module running on a computer.

[0106] The target prediction module 40 is configured to input the on-site image into a target detection model to perform target prediction when the on-site weight does not match the pre-stored weight, to obtain the bounding box information of the package in the on-site image, and to identify the number of packages according to the bounding box information. The target prediction module 40 can be a functional module running on a computer.

[0107] In this embodiment, the image acquisition module 10, the weight acquisition module 20, the search module 30, and the target prediction module 40 correspond to steps S10-S40 in Embodiment One, and can be understood by referring to Embodiment One.

[0108] As a preferred embodiment, the embodiment can further include the following functional modules:

[0109] The cropping module 50 is configured to crop the on-site image based on the bounding box information to obtain a package image when the number of packages identified by the bounding box information is one. The cropping module 50 can be a functional module running on a computer.

[0110] The classification prediction module 60 is configured to input the package image into a classification detection model to perform classification prediction, and to obtain a classification result of the number of packages as one or more. The classification prediction module 60 can be a functional module running on a computer.

[0111] In this embodiment, the cropping module 50 and the classification prediction module 60 correspond to steps S50-S60 in Embodiment Two, and can be understood by referring to Embodiment Two.

[0112] In summary, the above embodiments can achieve number detection of most stacked packages, thereby helping to determine the accurate weight of the packages and helping the logistics company or the customer to reduce the loss of shipping fees.

[0113] The above only describes part of the present application in detail, and the purpose is to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A method for detecting the number of packages, characterized in that: The following steps are involved: A1: Obtaining on-site images and on-site weights of a group of parcels; wherein the on-site images refer to images automatically captured on the parcel sorting line of a group of parcels currently being weighed; and the on-site weight refers to the weight obtained by weighing the group of parcels; A2: Searching for a corresponding pre-stored weight based on the package identification number in the on-site image; wherein the pre-stored weight refers to the weight data uploaded by the courier after weighing; A3: If the on-site weight does not match the pre-stored weight, the on-site image is input into the target detection model for target prediction to obtain bounding box information of the package in the on-site image; the bounding box information of the package can be used to determine whether the packages are stacked; A4: Identify the number of packages based on the bounding box information.

2. The method for detecting the number of packages according to claim 1, wherein: After step A4, the following steps are also included: A5: If the number of packages is identified as a single piece according to the bounding box information, the scene image is cropped based on the bounding box information to obtain a package image; A6: Input the package image into the classification detection model for classification prediction, and obtain the corresponding classification result of the number of packages as single or multiple.

3. The method for detecting the number of packages according to claim 2, wherein: The bounding box information includes a result vector; the result vector includes coordinate values, confidence values, and type values ​​related to the package.

4. The method for detecting the number of packages according to any one of claims 1 to 3, wherein: The package identification number is a symbol, number or identifying information that can distinguish the package from other packages.

5. The method for detecting the number of packages according to any one of claims 1 to 3, characterized in that After training, the target detection model is used to perform target prediction in step A3; the training of the target detection model includes the following steps: B1: Label the bounding box information of the package in the live image used for training; B2: Export the bounding box information into a labeling file that is compatible with the format required by the adopted training framework; B3: Convert the annotated file into a standard file and add it to the training set; Training is performed using the target detection algorithm and the training set based on the training framework.

6. The method for detecting the number of packages according to claim 5, wherein , before step B1, it also includes: B0: Perform target prediction on the training scene image through the target detection model to obtain the bounding box information wrapped in the training scene image.

7. The method for detecting the number of packages according to any one of claims 2 to 3, characterized in that After training, the classification detection model is used to perform classification prediction in step A6; the training of the classification detection model includes the following steps: C1: The scene image output after the target detection model performs target prediction is cropped according to the bounding box of the package to obtain the training package image; C2: storing the training package images in one of the single-item and multiple-item image sets according to the number of packages; C3: The deep learning classification algorithm is trained using the two image sets.

8. A package quantity detection device, characterized in that: Includes the following functional modules: Image acquisition module: used to obtain on-site images of a group of parcels; wherein, the on-site images refer to images automatically acquired from a group of parcels currently being weighed on the parcel sorting line; Weight collection module: used to obtain the on-site weight of the group of packages; the on-site weight refers to the weight obtained by weighing the group of packages; Search module: used to match the corresponding pre-stored weight according to the package identification number in the on-site image; wherein the pre-stored weight refers to the weight data uploaded by the courier after weighing; Target prediction module: used to input the on-site image into the target detection model for target prediction when the on-site weight does not match the pre-stored weight, obtain the bounding box information of the package in the on-site image; and identify the number of packages based on the bounding box information; the bounding box information of the package can be used to determine whether the packages are stacked.

9. The package quantity detection device according to claim 8, characterized in that: Also includes: A cropping module: configured to crop the scene image based on the bounding box information to obtain a package image when the bounding box information identifies that the number of packages is a single piece; Classification prediction module: used to input the package image into the classification detection model for classification prediction, and correspondingly obtain the quantity classification result of the package as single or multiple pieces.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the package quantity detection method according to any one of claims 1 to 7.

Images