A logistics parcel detection method based on a lightweight object detection model

Abstract

This invention discloses a logistics package detection method based on a lightweight object detection model. The model consists of four parts: an input module, a backbone module, a neck module, and a prediction module. The input module is the input end for image data, processing 752×480 three-channel image data collected on the logistics transportation line into 640×640×3 image data; the backbone module is used to extract logistics package features from the image data output by the input module; the neck module is used to mix and combine the logistics package features extracted by the backbone module; and the prediction module is used to process the image features obtained by the neck module and predict and output the target category and bounding box. This invention can meet the requirements of real-time performance, accuracy, lightweightness, and easy deployment in real working environments.

Description

Technical Field

[0001] The present invention belongs to the fields of logistics sorting and computer vision, and relates to a lightweight deep learning model for logistics package target detection. Background Art

[0002] During the sorting process, parcels may overlap, making it difficult to identify the labels of underlying parcels and causing them to be routed incorrectly. Furthermore, common parcels, such as bagged parcels, box parcels, envelope parcels, and bubble-wrapped parcels, need to be distinguished to provide appropriate grasping and sorting strategies. Therefore, identification of overlapping parcels, bagged parcels, box parcels, envelope parcels, and bubble-wrapped parcels is necessary. Current identification methods rely on human observation, such as surveillance footage, which is labor-intensive and error-prone. Other machine vision object detection methods include YOLOv3 combined with Deep-Sort (multi-object tracking) and pruning, and Faster R-CNN-based methods that improve the loss function to make candidate boxes closer to the target box. Because YOLOv3 has a larger number of parameters than YOLOv5, the model takes longer to train and detect. Faster R-CNN-based methods are two-stage detection methods, which suffer from poor real-time performance and are large and difficult to deploy. Summary of the Invention

[0003] In order to solve the problems in the current express package type identification process, such as the large number of parameters and calculations of the target detection model, long training time, poor real-time detection, and difficulty in deployment on embedded hardware, the present invention provides a logistics package detection method based on a lightweight target detection model, which can meet the requirements of real-time performance, accuracy, lightweight and easy deployment in a real working environment.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A logistics package detection method based on a lightweight object detection model is proposed. The model consists of four parts: input module, backbone module, neck module, and prediction module.

[0006] The input module is the input end of the image data, which is used to process the 3-channel image data with a size of 752×480 collected on the logistics transportation line into image data of 640×640×3;

[0007] The backbone module is used to extract the logistics package features of the image data output by the input module;

[0008] The neck module is used to mix and combine the logistics package features extracted by the backbone module;

[0009] The prediction module is used to process the image features obtained by the neck module, and predict and output the target category and bounding box, that is, use CIOU_Loss to perform backpropagation, update the gradient, and reduce the loss until the specified number of training times is reached or the Loss curve no longer decreases and the accuracy no longer improves, and use NMS to filter the target box;

[0010] The backbone module consists of layers 0 to 6, the neck module consists of layers 7 to 18, and the 15th and 18th layers serve as the medium target detection layer and large target detection layer of the prediction module, respectively.

[0011] The 0th layer is the CBRM layer. The 640×640×3 image data output by the input module is used as the input of the 0th layer, and the 160×160×8 image data output by the 0th layer is used as the input of the 1st layer.

[0012] The first layer is a Shuffle-Block layer with a step size of s=2. The input image data size of the first layer is 160×160×8, and the 80×80×64 image data output by the first layer is used as the input of the second layer.

[0013] The second layer is a C3-Lite layer, which uses three Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the second layer is 80×80×64. The 80×80×64 image data output by the second layer serves as the input of both the third and 13th layers.

[0014] The third layer is a Shuffle-Block layer with a step size of s=2. The image data size of the third layer input is 80×80×64, and the 40×40×128 image data output by the third layer is used as the input of the fourth layer.

[0015] The fourth layer is a C3-Lite layer, which uses 7 Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the fourth layer is 40×40×128. The 40×40×128 image data output by the fourth layer serves as the input of both the fifth and ninth layers.

[0016] The fifth layer is a Shuffle-Block layer with a step size of s=2. The input image data size of the fifth layer is 40×40×128, and the 20×20×256 image data output by the fifth layer is used as the input of the sixth layer.

[0017] The sixth layer is a C3-Lite layer, which uses three Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the sixth layer is 20×20×256. The 20×20×256 image data output by the sixth layer is used as the input of the seventh layer.

[0018] The 7th layer is a Conv layer. The input image data size of the 7th layer is 20×20×256. The 20×20×128 image data output by the 7th layer serves as the input of both the 8th and 17th layers.

[0019] The eighth layer is upsampling, and the input image data size of the eighth layer is 20×20×128. The 40×40×128 image data output by the eighth layer is used as the input of the ninth layer.

[0020] The 9th layer is a Concat layer, the input image size of the 9th layer is 40×40×128+40×40×128, and the 40×40×256 image data output by the 9th layer is used as the input of the 10th layer;

[0021] The 10th layer is the C3 layer, the input image size of the 10th layer is 40×40×256, and the 40×40×128 image data output by the 10th layer is used as the input of the 11th layer;

[0022] The 11th layer is a Conv layer, the input image size of the 11th layer is 40×40×128, and the 40×40×64 image data output by the 11th layer is used as the input of the 12th layer;

[0023] The 12th layer is an upsampling layer. The input image size of the 12th layer is 40×40×64. The 80×80×64 image data output by the 12th layer is used as the input of the 13th layer.

[0024] The 13th layer is a Concat layer. The input image size of the 13th layer is 80×80×64+80×80×64. The 80×80×128 image data output by the 13th layer is used as the input of the 14th layer.

[0025] The 14th layer is the Focus layer. The input image size of the 14th layer is 80×80×128. The 40×40×128 image data output by the 14th layer is used as the input of the 15th layer.

[0026] The 15th layer is the C3 layer. The input image size of the 15th layer is 40×40×128. The 40×40×64 image data output by the 15th layer is used as the input of the 16th layer. The 15th layer is the target detection layer of the prediction module.

[0027] The 16th layer is the Focus layer. The input image size of the 16th layer is 40×40×64. The 20×20×64 image data output by the 16th layer is used as the input of the 17th layer.

[0028] The 17th layer is a Concat layer. The input image size of the 17th layer is 20×20×64+20×20×128. The 20×20×192 image data output by the 17th layer is used as the input of the 18th layer.

[0029] The 18th layer is the C3 layer, the input image size of the 18th layer is 20×20×192, the output of the 18th layer is 20×20×64, and the 18th layer is the large object detection layer of the prediction module.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] Compared with the existing express parcel target detection model, the logistics parcel target detection model of the present invention significantly reduces the number of parameters and calculations and shortens the model training time while ensuring high recognition accuracy and high mAP value. The target detection speed is also better than most network models, and can be more easily deployed in the embedded hardware of sorting equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the Shuffle-YOLOV5 network structure;

[0033] Figure 2 It is a Shuffle-Block structure;

[0034] Figure 3 It is a C3-Lite structure;

[0035] Figure 4 This is the recognition effect of Shuffle-YOLOV5;

[0036] Figure 5 This is the test F1-Score value graph of Shuffle-YOLOV5;

[0037] Figure 6 This is the test accuracy graph of Shuffle-YOLOV5;

[0038] Figure 7 This is the test mAP graph of Shuffle-YOLOV5. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described below with reference to the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention that does not depart from the spirit and scope of the technical solution of the present invention should be included in the scope of protection of the present invention.

[0040] In response to the shortcomings of current common logistics package target detection methods, the present invention provides a logistics package detection method based on a lightweight target detection model, including: designing a lightweight feature extraction network to reduce the amount of parameter calculation and improve the feature extraction speed; designing a fusion method to fuse low-level fine-grained information with information in FPN+PAN (top-down + bottom-up feature pyramid network); using Focus instead of ordinary convolutional downsampling layers to reduce information loss and improve detection accuracy.

[0041] like Figure 1 As shown in the figure, the logistics package target detection model consists of four parts: input module, backbone module, neck module and prediction module, among which:

[0042] The input module is the input end of image data. Most of the image data of the present invention is collected in real scenes, namely logistics transportation lines, and is a three-channel image with a size of 752×480. The image data enters the input module and is uniformly processed into image data with a size of 640×640×3 (number of channels) through operations such as Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling.

[0043] The backbone module is used to extract the logistics package features of the image data output by the input module, and the 640×640×3 image data output by the input module is used as the input of the 0th layer CBRM. CBRM (abbreviation of Conv, BN, ReLU, Maxpooling) refers to a convolution operation with a size of 3×3 and a stride of 2, with an output of 320×320×8 and a maximum pooling operation with a size of 3×3 and a stride of 2, with an output of 160×160×8. The input of the first layer Shuffle-Block (step: s=2) comes from the output of the 0th layer CBRM, and the output is 80×80×64. The second layer C3-Lite (step: s=1) uses 3 Bottleneck structures, with an input of 80×80×64 from the output of the first layer Shuffle-Block, and an output of 80×80×64 from the second layer. The output of the second layer acts on the third and 13th layers at the same time. Huffle-Block and C3-Lite are the main unit modules of the lightweight backbone designed by this invention. The input of the Shuffle-Block (step size: s=2) of the third layer comes from the output 80×80×64 of the second layer, and the output is 40×40×128. The C3-Lite (step size: s=1) of the fourth layer uses 7 Bottleneck structures. The input of the fourth layer comes from the output 40×40×128 of the third layer, and the output is 40×40×128. The output of the fourth layer acts on the fifth and ninth layers simultaneously. The input of the Shuffle-Block (step size: s=2) of the fifth layer comes from the output 40×40×128 of the fourth layer, and the output is 20×20×256. The C3-Lite (step size: s=1) of the sixth layer uses 3 Bottleneck structures. The input of the sixth layer comes from the output 20×20×256 of the fifth layer, and the output is 20×20×256.

[0044] The neck module is used to mix and combine the logistics package features extracted by the backbone module. The input of the 7th layer Conv comes from the output of the 6th layer (20×20×256), and the output is 20×20×128. It acts on the 8th and 17th layers at the same time. The input of the 8th layer upsampling comes from the output of the 7th layer (20×20×128), and the output is 40×40×128. The input of the 9th layer Concat comes from the output of the 8th layer (40×40×128). The output of the 4th layer is 40×40×128. The two are concat-operated according to the number of channels, and the output is 40×40×256. The input of the 10th layer C3 comes from the output of the 9th layer 40×40×256, and the output is 40×40×128. The input of the 11th layer Conv comes from the output of the 10th layer 40×40×128, and the output is 40×40×64. The input of the 12th layer upsampling comes from the output of the 11th layer 40×40×64, and the output is 8 0×80×64, the input of the 13th Concat layer comes from the output of the 12th layer 80×80×64 and the output of the 2nd layer 80×80×64, and the output is 80×80×128. The input of the 14th Focus layer comes from the output of the 13th layer 80×80×128, and the output is 40×40×128. The input of the 15th C3 layer comes from the output of the 14th layer 40×40×128, and the output is 40×40×64. The 15th layer is the target detection layer The input of the 16th layer Focus comes from the output 40×40×64 of the 15th layer, and the output is 20×20×64. The input of the 17th layer Concat comes from the output 20×20×64 of the 16th layer and the output of the 7th layer, which is 20×20×128. The output is 20×20×192 according to the number of channels. The input of the 18th layer comes from the output 20×20×192 of the 17th layer, and the output is 20×20×64. The 18th layer is the large target detection layer.

[0045] The 15th and 18th layers serve as the medium target detection layer and large target detection layer of the prediction module, respectively. The prediction module is used to process the image features obtained by the neck module, and predict and output the target category and bounding box. That is, CIOU_Loss (loss function) is used for backpropagation, gradient update, and loss reduction until the specified number of training times is reached or the Loss curve no longer decreases and the accuracy no longer improves. NMS (non-maximum suppression) is used to filter the target box.

[0046] The specific design method is as follows:

[0047] 1. Backbone design: a lightweight feature extraction network that can more effectively ensure information exchange between groups.

[0048] The backbone of existing express package detection models contains numerous convolution operations, which increase the model's parameter count and computational complexity. This paper designs a lightweight backbone, whose main unit modules are named Shuffle-Block and C3-Lite, which can significantly reduce the number of parameters and computational complexity.

[0049] like Figure 2 As shown, Shuffle-Block consists of branch 1, branch 2 and Channel Shuffle, where the number of channels of branch 1 and branch 2 is the same as the number of feature channels of the input, and branch 1 is shuffled by 3 3 / 2 DWConv, 1 1Conv is processed; branch 2 passes 1 1Conv, 3 3 / 2 DWConv, 1 1Conv is used for processing; since the number of channels of branch 1 and branch 2 is the same as the number of feature channels of the input, branches 1 and branch 2 are added according to the number of channels after Concat processing, the length and width of the image data are halved, and the number of channels is doubled. The output of Concat is used as the input of Channel Shuffle. The DWConv and GConv (Group Conv) used in Channel Shuffle are both lighter convolution operations than conventional convolution.

[0050] like Figure 3 As shown, C3-Lite consists of branch 1 and branch 2, where the feature map input branch 1 passes through 1 After the DWConv convolution operation of 1 is reduced in dimension, it is then subjected to n Bottleneck superposition operations (n is 3, 7, and 3 respectively), and the dimension remains unchanged. The feature map obtained is the same as that of branch 2 after 1 The feature map obtained by the DWconv convolution operation of 1 is subjected to the Concat operation, and finally passed through 1 The final output of the CBS standard convolution operation of 1 is obtained. This design reduces the number of parameters of the model. Bottleneck consists of branches a and b, where branch b does not perform any processing and branch a first uses 1 1 standard convolution for dimensionality reduction, and then 3 The standard convolution of 3 is used to increase the dimension, and then the Add operation is performed with branch b, and the dimension remains unchanged.

[0051] 2. Based on the characteristics of express package detection images, a multi-scale feature fusion method including medium and large targets is designed.

[0052] The present invention performs target recognition of logistics packages in a specific equipment environment. Therefore, various packages are medium targets or large targets in the image. Therefore, the prediction part of the designed neural network model includes two parts: medium target detection and large target detection. That is, 40×40×64 in the Prediction module is used to detect medium targets, and 20×20×64 is used to detect large targets.

[0053] 3. Design a method to fuse low-level fine-grained information with information in FPN+PAN to fully utilize fine-grained features.

[0054] To rationally utilize shallow feature information in feature maps, the present invention devised a method that fuses low-level, fine-grained information with information from FPN+PAN (see Table 1). This method derives the features of the mid-range target after feature fusion. This method feeds the feature map after passing through the fourth C3-Lite layer to the ninth Concat layer, the feature map after passing through the second C3-Lite layer to the thirteenth Concat layer, and the feature map after passing through the seventh Conv layer to the seventeenth Concat layer. This feature fusion approach not only increases the speed of the neural network but also ensures full utilization of low-level, fine-grained information, thereby enhancing the effectiveness of the neural network.

[0055] Table 1

[0056]

[0057] Fourth, design two Focus layers for downsampling to reduce information loss caused by downsampling and improve accuracy and mAP.

[0058] Since the downsampling layer in the existing express package target detection model is implemented by ordinary convolution, this implementation method will cause information loss. To address this problem, the present invention designs two Focus layers for downsampling. The high-resolution feature map is split into multiple low-resolution images / feature maps through slicing operations. Combining the feature fusion method of the three designs, the Focus layer converts the information on the width and height planes into channel dimensions, and then through 3 3. Convolution is used to extract different features. This method can reduce the information loss caused by downsampling and improve the accuracy and mAP of the model.

[0059] Example:

[0060] (1) Build the operating environment of the logistics package target detection model of the present invention. The hardware configuration is a 12th Gen Intel Core i9-12900K 3020GHz processor, an NVIDIA GeForce GTX 3090Ti graphics card, an operating system is a Windows 11 64-bit system, a deep learning framework is Pytorch, the GPU acceleration library is CUDA 11.3 and CUDNN 8.3.2, and the programming language is Python (software and hardware are not limited to the above configuration environment).

[0061] (2) Dataset construction and data enhancement. The first step is to obtain image data. Most of the image data for this task were taken in a real working environment, namely, on a logistics transportation line. Data enhancement was performed using methods such as image flipping, brightness and color enhancement, and contrast enhancement. The dataset was then labeled and randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0062] (3) The training set and validation set are sent to the pre-training model for model training. When the specified number of training times is reached or the loss curve no longer decreases and the accuracy no longer increases, the trained model is saved, and then the test set is sent to the saved network model to output the target detection results of the logistics package. The detection results are as follows: Figures 4 to 7 shown.

[0063] pass Figures 4-7 It can be concluded that under the evaluation indicators of target detection such as precision and mAP, it can be seen that the precision, recall rate and mAP of the present invention are all excellent, thereby proving the effectiveness of the present invention.

Claims

1. A logistics package detection method based on a lightweight target detection model, characterized by The model consists of four parts: input module, backbone module, neck module and prediction module, among which: The input module is the input end of the image data, which is used to process the 3-channel image data with a size of 752×480 collected on the logistics transportation line into image data of 640×640×3; The backbone module is used to extract the logistics package features of the image data output by the input module; The neck module is used to mix and combine the logistics package features extracted by the backbone module; The prediction module is used to process the image features obtained by the neck module, and predict and output the target category and bounding box; The backbone module consists of layers 0 to 6, the neck module consists of layers 7 to 18, and the 15th and 18th layers serve as the medium target detection layer and large target detection layer of the prediction module, respectively. The 0th layer is the CBRM layer. The 640×640×3 image data output by the input module is used as the input of the 0th layer, and the 160×160×8 image data output by the 0th layer is used as the input of the 1st layer. The first layer is a Shuffle-Block layer with a step size of s=2. The input image data size of the first layer is 160×160×8, and the 80×80×64 image data output by the first layer is used as the input of the second layer. The second layer is a C3-Lite layer, which uses three Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the second layer is 80×80×64. The 80×80×64 image data output by the second layer serves as the input of both the third and 13th layers. The third layer is a Shuffle-Block layer with a step size of s=2. The image data size of the third layer input is 80×80×64, and the 40×40×128 image data output by the third layer is used as the input of the fourth layer. The fourth layer is a C3-Lite layer, which uses 7 Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the fourth layer is 40×40×128. The 40×40×128 image data output by the fourth layer serves as the input of both the fifth and ninth layers. The fifth layer is a Shuffle-Block layer with a step size of s=2. The input image data size of the fifth layer is 40×40×128, and the 20×20×256 image data output by the fifth layer is used as the input of the sixth layer. The sixth layer is a C3-Lite layer, which uses three Bottleneck structures. The step size of the C3-Lite layer is s=1. The input image data size of the sixth layer is 20×20×256. The 20×20×256 image data output by the sixth layer is used as the input of the seventh layer. The 7th layer is a Conv layer. The input image data size of the 7th layer is 20×20×256. The 20×20×128 image data output by the 7th layer is used as the input of the 8th and 17th layers at the same time. The eighth layer is upsampling, and the input image data size of the eighth layer is 20×20×128. The 40×40×128 image data output by the eighth layer is used as the input of the ninth layer. The 9th layer is a Concat layer, the input image size of the 9th layer is 40×40×128+40×40×128, and the 40×40×256 image data output by the 9th layer is used as the input of the 10th layer; The 10th layer is the C3 layer, the input image size of the 10th layer is 40×40×256, and the 40×40×128 image data output by the 10th layer is used as the input of the 11th layer; The 11th layer is a Conv layer, the input image size of the 11th layer is 40×40×128, and the 40×40×64 image data output by the 11th layer is used as the input of the 12th layer; The 12th layer is an upsampling layer. The input image size of the 12th layer is 40×40×64. The 80×80×64 image data output by the 12th layer is used as the input of the 13th layer. The 13th layer is a Concat layer. The input image size of the 13th layer is 80×80×64+80×80×64. The 80×80×128 image data output by the 13th layer is used as the input of the 14th layer. The 14th layer is the Focus layer. The input image size of the 14th layer is 80×80×128. The 40×40×128 image data output by the 14th layer is used as the input of the 15th layer. The 15th layer is the C3 layer. The input image size of the 15th layer is 40×40×128. The 40×40×64 image data output by the 15th layer is used as the input of the 16th layer. The 15th layer is the target detection layer of the prediction module. The 16th layer is the Focus layer. The input image size of the 16th layer is 40×40×64. The 20×20×64 image data output by the 16th layer is used as the input of the 17th layer. The 17th layer is a Concat layer. The input image size of the 17th layer is 20×20×64+20×20×128. The 20×20×192 image data output by the 17th layer is used as the input of the 18th layer. The 18th layer is the C3 layer, the input image size of the 18th layer is 20×20×192, the output of the 18th layer is 20×20×64, and the 18th layer is the large object detection layer of the prediction module.

2. The logistics package detection method based on a lightweight target detection model according to claim 1 is characterized in that The Shuffle-Block consists of branch 1, branch 2 and Channel Shuffle, where the number of channels in branch 1 and branch 2 is the same as the number of input feature channels, and branch 1 is shuffled by 3 3 / 2 DWConv, 1 1Conv is processed; branch 2 passes 1 1Conv, 3 3 / 2 DWConv, 1 1Conv is used for processing; after the Concat processing, branches 1 and 2 are added according to the number of channels. The length and width of the image data are halved, and the number of channels is doubled. The output of Concat is used as the input of Channel Shuffle.

3. The logistics package detection method based on a lightweight target detection model according to claim 1 is characterized in that The C3-Lite layer consists of branch 1 and branch 2. The feature map input branch 1 passes through 1 After the DWConv convolution operation of 1 is reduced in dimension, the dimension remains unchanged after the Bottleneck superposition operation. The obtained feature map is the same as that of branch 2 after 1 The feature map obtained by the DWconv convolution operation of 1 is subjected to the Concat operation, and finally passed through 1 The final output of the CBS standard convolution operation of 1.

4. The logistics package detection method based on a lightweight target detection model according to claim 3 is characterized in that The Bottleneck consists of branches a and b, wherein: branch b does not perform any processing, branch a first uses 1 1 standard convolution for dimensionality reduction, and then 3 The standard convolution of 3 is used to increase the dimension, and then the Add operation is performed with branch b, and the dimension remains unchanged.

Images