Reflux identification method, device, equipment and storage medium

Abstract

The present application relates to the technical field of visual monitoring, and discloses a backflow component identification method, device, equipment and storage medium. The method comprises the following steps: acquiring a first image in which a backflow component detection area exists and a second image in which the backflow component detection area does not exist, labeling the first image and the second image as training sample images, and obtaining labeling information of each training sample image; training a preset SSD-Pytorch model by using the training sample images and the labeling information, and obtaining a backflow component identification model; acquiring a third image of a current backflow component detection area, inputting the third image into the backflow component identification model for identification, and outputting a probability of the third image existing the backflow component; and if the probability is greater than a preset probability threshold, it is determined that the third image exists the backflow component and a prompt information is pushed. The present application improves the identification accuracy of the logistics backflow component, reduces the safety hidden danger caused by the backflow component blocking the logistics assembly line, and increases the efficiency of express delivery.

Description

Technical Field

[0001] This invention relates to the field of visual monitoring technology, and in particular to a method, apparatus, device, and storage medium for identifying return parts. Background Technology

[0002] The express delivery industry handles a massive number of parcels daily, and return shipments are a frequent occurrence for various reasons. This leads to backlogs of returned packages, cluttered and disorganized goods, and potential safety hazards. To ensure timely and efficient delivery to customers and prevent returns, video surveillance technology can be used to analyze whether return shipments have occurred, allowing delivery personnel to manage them systematically.

[0003] Current video surveillance technology for return shipments employs the NMS (Non-Maximum Suppression) algorithm. This algorithm has significant drawbacks. First, NMS requires a hyperparameter; a hyperparameter that is too large or too small is problematic, making it difficult to balance. Second, NMS removes one of two adjacent high-probability bounding boxes, leading to missed detections. Furthermore, the loss function applied by the algorithm heavily emphasizes easily classified samples, making it difficult for the model to focus on difficult-to-classify samples during training. Consequently, existing video surveillance technology has low accuracy in identifying return shipments. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problem of low accuracy in identifying return shipments in existing video surveillance technology.

[0005] The first aspect of the present invention provides a method for identifying return parts, comprising:

[0006] Acquire a first image of the pre-collected return component detection area containing a return component and a second image of the area where a return component does not exist, and use the first image and the second image as training sample images;

[0007] The training sample images are labeled to obtain the labeling information for each training sample image;

[0008] Using the training sample images and the annotation information, a pre-set SSD-Pytorch model is trained to obtain a return component recognition model. The SSD-Pytorch model includes a VGG Backbone layer, Extra Layers layers, and Multi-box Layers layers.

[0009] Obtain the third image of the current return component detection area, input the third image into the return component recognition model for recognition, and output the probability that a return component exists in the third image;

[0010] If the probability is greater than a preset probability threshold, then it is determined that a return component exists in the third image and a prompt message is pushed.

[0011] Optionally, in a first implementation of the first aspect of the present invention, the step of annotating the training sample images to obtain annotation information for each training sample image includes:

[0012] The region of the reflow component in the training sample image of the pre-collected reflow component detection area is delineated, and the training sample image is segmented to obtain the corresponding labeled region;

[0013] The labeled area is written into a pre-set blank XML file to obtain the corresponding labeling information.

[0014] Optionally, in a second implementation of the first aspect of the present invention, the step of training a preset SSD-PyTorch model using the training sample images and the annotation information to obtain a return component identification model includes:

[0015] The training sample image is input into the VGG Backbone layer, and the initial feature map of the training sample image is extracted through the VGG Backbone layer;

[0016] Each initial feature map is input into the Extra Layers layer, and the initial feature maps are downsampled by the Extra Layers layer to obtain the corresponding feature maps;

[0017] The feature map is input into the Multi-box Layers layer, and the feature map is convolved through the Multi-box Layers layer to obtain the corresponding regression feature map and classification feature map;

[0018] Based on the regression feature map and the classification feature map, predict the probability that a reflow component exists in the training sample image;

[0019] Based on the probability and the annotation information, the parameters of the SSD-Pytorch model are adjusted until the SSD-Pytorch model converges, thus obtaining the return component identification model.

[0020] Optionally, in a third implementation of the first aspect of the present invention, predicting the probability of a reflow component existing in the training sample image based on the regression feature map and the classification feature map includes:

[0021] By integrating the regression feature map and the classification feature map, prediction boxes for multiple target objects in the training sample image are determined, and the initial probability that the target object in each prediction box is a reflow part is determined.

[0022] The initial probabilities are sorted in descending order to determine the predicted bounding boxes of each target object in the training sample image;

[0023] Based on the sorting order of the prediction boxes, prediction boxes with an initial probability greater than a preset probability threshold are selected as candidate boxes, and the initial probabilities of prediction boxes adjacent to the candidate boxes are adjusted to obtain the final probability that each target in the training sample image is a reflow component.

[0024] Based on the final probability that each target object is a reflow component, the probability that a reflow component exists in the training sample image is determined.

[0025] Optionally, in a fourth implementation of the first aspect of the present invention, the step of selecting prediction boxes with initial probabilities greater than a preset probability threshold as candidate boxes based on the sorting order of the prediction boxes, and adjusting the initial probabilities of prediction boxes adjacent to the candidate boxes to obtain the final probability that each target object in the training sample image is a reflow component includes:

[0026] Select the highest-ranked predicted bounding box as the current candidate bounding box, and calculate the IOU value between the current candidate bounding box and the adjacent predicted bounding boxes.

[0027] Based on the IOU value, the initial probability of the adjacent prediction box is reduced, and the reduced probability is used as the final probability;

[0028] Remove the predicted boxes that are candidate boxes and the predicted boxes after adjusting the initial probability, and reselect the predicted box that ranks first as the current candidate box.

[0029] The process jumps to the step of calculating the IOU value between the current candidate box and the adjacent predicted box, and stops when the initial probability of the latest selected candidate box is less than the preset probability threshold, thus obtaining the final probability that each target object in the training sample image is a reflow component.

[0030] Optionally, in a fifth implementation of the first aspect of the present invention, adjusting the parameters of the SSD-Pytorch model according to the probability and the annotation information until the SSD-Pytorch model converges to obtain the return component identification model includes:

[0031] Based on the probability and the annotation information, calculate the cross-entropy loss value of the SSD-Pytorch model, and determine whether the cross-entropy loss value is less than a preset loss threshold.

[0032] If the cross-entropy loss value is less than the preset loss threshold, the parameters of the SSD-Pytorch model are adjusted and retrained until the cross-entropy loss value is less than the preset loss threshold. The SSD-Pytorch model is then considered to have converged, and the return component identification model is obtained.

[0033] A second aspect of the present invention provides a return component identification device, comprising:

[0034] The acquisition module is used to acquire a first image of the detection area of ​​the return component where there is a return component and a second image of the detection area where there is no return component, and to use the first image and the second image as training sample images;

[0035] The annotation module is used to annotate the training sample images to obtain the annotation information of each training sample image;

[0036] The training module is used to train a pre-set SSD-Pytorch model using the training sample images and the annotation information to obtain a return component recognition model. The SSD-Pytorch model includes a VGG Backbone layer, an ExtraLayers layer, and a Multi-box Layers layer.

[0037] The recognition module is used to acquire a third image of the current return component detection area, input the third image into the return component recognition model for recognition, and output the probability that a return component exists in the third image;

[0038] The push module is used to determine that a return component exists in the third image and push a prompt message if the probability is greater than a preset probability threshold.

[0039] Optionally, in a first implementation of the first aspect of the present invention, the annotation module includes:

[0040] The segmentation unit is used to delineate the region of the reflow component in the training sample image of the pre-collected reflow component detection area, and to segment the training sample image to obtain the corresponding labeled region;

[0041] The writing unit is used to write the labeled area into a pre-set blank XML format file to obtain the corresponding labeling information.

[0042] Optionally, in a second implementation of the first aspect of the present invention, the training module includes:

[0043] The first extraction unit is used to input the training sample image into the VGG Backbone layer and extract the initial feature map of the training sample image through the VGG Backbone layer.

[0044] The downsampling unit is used to input the initial feature maps into the Extra Layers layer, and downsample the initial feature maps through the Extra Layers layer to obtain the corresponding feature maps;

[0045] A convolutional unit is used to input the feature map into the Multi-box Layers layer and perform convolution processing on the feature map through the Multi-box Layers layer to obtain the corresponding regression feature map and classification feature map;

[0046] The prediction unit is used to predict the probability that a return component exists in the training sample image based on the regression feature map and the classification feature map.

[0047] The adjustment unit is used to adjust the parameters of the SSD-Pytorch model according to the probability and the annotation information until the SSD-Pytorch model converges to obtain the return component identification model.

[0048] Optionally, in a third implementation of the first aspect of the present invention, the prediction unit includes:

[0049] An integration subunit is used to integrate the regression feature map and the classification feature map to determine the prediction boxes of multiple target objects in the training sample image, and to determine the initial probability that the target object in each prediction box is a reflow part;

[0050] The sorting subunit is used to sort the predicted bounding boxes of each target object in the training sample image according to the initial probabilities in descending order;

[0051] The adjustment subunit is used to select prediction boxes with an initial probability greater than a preset probability threshold as candidate boxes based on the sorting order of the prediction boxes, and adjust the initial probability of prediction boxes adjacent to the candidate boxes to obtain the final probability that each target object in the training sample image is a reflow component.

[0052] A sub-unit is defined to determine the probability that a reflow component exists in the training sample image based on the final probability that each target object is a reflow component.

[0053] Optionally, in a fourth implementation of the first aspect of the present invention, the adjustment subunit is further configured to:

[0054] Select the highest-ranked predicted bounding box as the current candidate bounding box, and calculate the IOU value between the current candidate bounding box and the adjacent predicted bounding boxes.

[0055] Based on the IOU value, the initial probability of the adjacent prediction box is reduced, and the reduced probability is used as the final probability;

[0056] Remove the predicted boxes that are candidate boxes and the predicted boxes after adjusting the initial probability, and reselect the predicted box that ranks first as the current candidate box.

[0057] The process jumps to the step of calculating the IOU value between the current candidate box and the adjacent predicted box, and stops when the initial probability of the latest selected candidate box is less than the preset probability threshold, thus obtaining the final probability that each target object in the training sample image is a reflow component.

[0058] Optionally, in a fifth implementation of the first aspect of the present invention, the adjustment unit includes:

[0059] The discrimination subunit is used to calculate the cross-entropy loss value of the SSD-Pytorch model based on the probability and the annotation information, and to determine whether the cross-entropy loss value is less than a preset loss threshold.

[0060] An iterative subunit is used to adjust the parameters of the SSD-Pytorch model and retrain it if the cross-entropy loss value is less than a preset loss threshold, until the cross-entropy loss value is less than the preset loss threshold, thus determining that the SSD-Pytorch model has converged and obtaining the return component recognition model.

[0061] A third aspect of the present invention provides a return item identification device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the return item identification device to perform the above-described return item identification method.

[0062] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described reflow identification method.

[0063] The technical solution provided by this invention uses training sample images of the return shipment detection area and corresponding annotation information as training samples. Then, the SSD-PyTorch model is trained using these training sample images and annotation information to obtain a corresponding return shipment recognition model, which is directly applied to the subsequent recognition of return shipments. Next, when a third image of the current return shipment detection area is acquired, it is input into the return shipment recognition model for identification to predict the probability of a return shipment existing in the third image. Finally, if the predicted probability is greater than a preset probability threshold, a return shipment is identified in the third image, and a notification is sent to assist staff in determining whether a return shipment has been generated. This invention improves the accuracy of return shipment recognition, reduces the safety hazards caused by return shipments blocking the logistics line, and increases the efficiency of express delivery. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of one embodiment of the return component identification method in this invention;

[0065] Figure 2This is a schematic diagram of another embodiment of the return component identification method in this invention;

[0066] Figure 3 This is a schematic diagram of one embodiment of the return component identification device in this invention;

[0067] Figure 4 This is a schematic diagram of another embodiment of the return component identification device in this invention;

[0068] Figure 5 This is a schematic diagram of one embodiment of the return component identification device in this invention. Detailed Implementation

[0069] This invention provides a method, apparatus, device, and storage medium for identifying return shipments. The method involves acquiring a first image showing the presence of a return shipment in a detection area and a second image showing the absence of a return shipment. These first and second images are then used as training samples and labeled to obtain annotation information for each training sample image. A pre-set SSD-PyTorch model is trained using the training sample images and annotation information to obtain a return shipment identification model. A third image of the current return shipment detection area is acquired and input into the return shipment identification model for identification. The probability of a return shipment being present in the third image is output. If the probability is greater than a pre-set probability threshold, a return shipment is confirmed to be present in the third image, and a notification is sent. This invention improves the accuracy of returning shipment identification, reduces the safety hazards caused by return shipments blocking logistics lines, and increases the efficiency of express delivery.

[0070] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0071] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the return component identification method in this invention includes:

[0072] 101. Obtain a first image of the pre-collected return component detection area where a return component exists and a second image of the area where a return component does not exist, and use the first image and the second image as training sample images;

[0073] It is understood that the executing entity of this invention can be a return device identification device, a terminal, or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0074] In this embodiment, a return component may or may not be present in the return component detection area. Therefore, the first image is an image collected when a return component is present in the return component detection area, and the second image is an image collected when a return component is not present in the return component detection area.

[0075] 102. Label the training sample images to obtain the labeling information for each training sample image;

[0076] In this embodiment, two methods are generally used to annotate the training sample images: model annotation and manual annotation. Since there is currently a lack of models capable of accurately annotating express parcels, this solution uses manual annotation. First, the training sample images are input into a pre-set image annotation software for display; then, the regions of return parcels in each training sample image are manually delineated to automatically generate annotation information. The specific steps are as follows:

[0077] (1) Delineate the region of the reflow component in the training sample image of the pre-collected reflow component detection area, and segment the training sample image to obtain the corresponding labeled region;

[0078] (2) Write the labeled area into a pre-set blank XML file to obtain the corresponding labeled information.

[0079] In this embodiment, LabelImg software is preferred as the image annotation software. A new folder is created beforehand to store training sample images and corresponding annotation information, including four folders: Annotations, ImageSet, JPEGImages, SegmentationClass, and SegmentationObject. JPEGImages stores the training sample images, and Annotations stores corresponding XML (Extensible Markup Language) annotation information with the same name. Note that the annotated category names use lowercase letters, such as using "car" instead of "Car". ImageSet stores the Main folder, which contains the txt files of the training sample images.

[0080] 103. Using the training sample images and the annotation information, train the preset SSD-Pytorch model to obtain the return part recognition model, wherein the SSD-Pytorch model includes a VGG Backbone layer, Extra Layers layer, and Multi-box Layers layer;

[0081] In this embodiment, the SSD-Pytorch (Single Shot MultiBox Detector-Pytorch) model is a one-stage model recognition method that can perform multi-box prediction. It can filter the prediction boxes of multiple targets in the training sample image through a convolutional neural network, and then perform classification and regression processing on the prediction boxes to directly identify the logistics packages within the prediction boxes, thereby determining whether the target in the corresponding prediction box is a return package or other target.

[0082] Specifically, the SSD-PyTorch model consists of a VGG Backbone layer, Extra Layers, and Multi-box Layers, detailed below:

[0083] A. For the VGG Backbone layer, the first five layers of VGG16 (Visual Geometry Group Network 16, Super-resolution Test Sequence 16) can be used as the base network to initially extract the initial feature map from the training sample images. The maximum number of channels in the initial feature map is 512*19*19.

[0084] B. The Extra Layers replace the pooling and connection layers in the VGG16 network and can contain 8 convolutional layers to further downsample the initial feature map, resulting in a final feature map with a maximum number of channels of 1024*19*19, which can be used for subsequent extraction of multi-scale prediction boxes based on the size of the third image.

[0085] C. For the Multi-box Layers, the feature maps extracted earlier have six different scales of express delivery recognition targets, in the following order: 512*38*38, 1024*19*19, 512*10*10, 256*5*5, 256*3*3, and 256*1*1. The feature maps of the earlier sizes predict larger targets, and the feature maps of the later sizes predict smaller targets. Each layer can be convolved with loc and conf respectively to obtain the corresponding class feature maps and regression feature maps.

[0086] 104. Obtain the third image of the current return component detection area, input the third image into the return component recognition model for recognition, and output the probability that a return component exists in the third image;

[0087] In this embodiment, a camera is placed in the reflow part detection area to monitor the reflow part detection area in real time. The camera captures a third image at regular intervals (e.g., in 0.1s increments) and inputs it into the reflow part recognition model. Based on the appearance characteristics of the reflow part, the reflow part is selected from multiple target objects in the third image.

[0088] In this embodiment, the training sample images are sequentially processed through a neural network consisting of a VGG Backbone layer, Extra Layers layer, and Multi-box Layers layer to extract features, ultimately obtaining a category feature map and a regression feature map. Finally, the Soft-NMS algorithm is used to determine the predicted bounding boxes of each target object in the training sample images and to determine the probability that the target object within each predicted bounding box is a reflow component.

[0089] To determine the bounding boxes, consider three overlapping bounding boxes for two objects with confidence scores of 0.9, 0.7, and 0.85. The highest-scoring bounding box is selected, and after the first processing, the scores become 0.9, 0.55, and 0.75 (this highest-scoring bounding box is saved). Then, the second highest-scoring bounding box is selected, and after processing, the confidence scores become 0.9, 0.45, and 0.7 (all three bounding boxes are now present). Finally, the third bounding box is selected, and its score remains unchanged. The final confidence scores of the three boxes after Soft-NMS suppression are 0.9, 0.45, and 0.7. Finally, a threshold is set to remove bounding boxes with scores below the preset threshold, leaving only the original bounding boxes.

[0090] 105. If the probability is greater than a preset probability threshold, then it is determined that there is a return component in the third image and a prompt message is pushed.

[0091] In this embodiment, when a return item is present in the third image, a prompt message can be sent to the business personnel to notify them that a return item has been generated and to dispatch the item as soon as possible. The prompt message can be text, voice, or light-based.

[0092] The technical solution provided by this invention uses training sample images of the return shipment detection area and corresponding annotation information as training samples. Then, the SSD-PyTorch model is trained using these training sample images and annotation information to obtain a corresponding return shipment recognition model, which is directly applied to the subsequent recognition of return shipments. Next, when a third image of the current return shipment detection area is acquired, it is input into the return shipment recognition model for identification to predict the probability of a return shipment existing in the third image. Finally, if the predicted probability is greater than a preset probability threshold, a return shipment is identified in the third image, and a notification is sent to assist staff in determining whether a return shipment has been generated. This invention improves the accuracy of return shipment recognition, reduces the safety hazards caused by return shipments blocking the logistics line, and increases the efficiency of express delivery.

[0093] Please see Figure 2 The second embodiment of the return component identification method in this invention includes:

[0094] 201. Input the training sample image into the VGG Backbone layer, and extract the initial feature map of the training sample image through the VGG Backbone layer;

[0095] In this embodiment, the VGG Backbone layer network structure is a deep learning network used for object localization and classification. Here, five convolutional layers of VGG16 are extracted, and its three fully connected layers and SoftMax output layer are removed. The five convolutional layers of VGG16 are used as the basic network structure, and multiple convolutional layers with smaller kernels (3*3) are used. On the one hand, this reduces the number of parameters; on the other hand, it is equivalent to performing more non-linear mapping, which can increase the network's fitting / expressive ability and improve the efficiency of extracting initial feature maps from training sample images and the accuracy of feature recognition. The size of the initial feature map includes 512*38*38.

[0096] 202. Input each initial feature map into the Extra Layers layer, and downsample the initial feature maps through the Extra Layers layer to obtain the corresponding feature maps;

[0097] In this embodiment, the Extra Layers layer adds multiple convolutional layers connected to the VGG Backbone layer to downsample the initial feature map, thereby increasing the range of feature map recognition sizes and enabling the recognition of larger third images. The Extra Layers layer may include two convolutional networks with sizes of 1024*3*3 and 1024*1*1, while the final feature map sizes relative to the initial feature map include: 1024*19*19, 512*10*10, 256*5*5, 256*3*3, and 256*1*1.

[0098] 203. Input the feature map into the Multi-box Layers layer, and perform convolution processing on the feature map through the Multi-box Layers layer to obtain the corresponding regression feature map and classification feature map;

[0099] In this embodiment, after the Extra Layers layer, there are a total of 6 multi-scale extracted feature maps. Here, the feature maps (loc and conf) can be further convolved by the Multi-box Layers layer, using a multi-scale branch regression network and a multi-scale branch classification network respectively, to finally obtain the regression feature map and classification feature map of each target prediction box in the third image.

[0100] 204. Based on the regression feature map and the classification feature map, predict the probability that a return component exists in the training sample image;

[0101] In this embodiment, the classification feature map can be represented by the output vector of the classification score of Softmax, and the regression feature map can be represented by the output vector of bounding-box window regression. Non-maximum suppression is applied to each class of objects in the window regression using the classification score to remove overlapping predicted boxes, ultimately obtaining the window with the highest score after regression correction in each class. The specific concept prediction process is as follows:

[0102] (1) Integrate the regression feature map and the classification feature map to determine the prediction boxes of multiple target objects in the training sample image, and determine the initial probability that the target object in each prediction box is a reflow part;

[0103] (2) Sort the prediction boxes of each target object in the training sample image according to the order of the initial probabilities from high to low;

[0104] (3) Based on the sorting order of the prediction boxes, select the prediction boxes with an initial probability greater than the preset probability threshold as candidate boxes, and adjust the initial probability of the prediction boxes adjacent to the candidate boxes to obtain the final probability of each target object in the training sample image being a reflow component.

[0105] (4) Based on the final probability that each target object is a reflow component, determine the probability that a reflow component exists in the training sample image.

[0106] In this embodiment, the process can use the Soft-NMS algorithm to predict the probability that the target object is a repatriated part. The algorithm is simple: sort the boxes according to their classification probabilities, and use the box with the highest probability as a candidate box. Then, for all predicted boxes whose IOU (Intersection-over-Union) with it is higher than a threshold (a pre-set hyperparameter), reduce their initial probability to obtain the final probability. Next, find the predicted box with the second highest probability among the remaining predicted boxes as a candidate box, and further reduce the initial probability of all other predicted boxes whose IOU with it is higher than a preset threshold. This process is repeated to obtain the probability of all predicted boxes. The specific steps are as follows:

[0107] (1) Select the highest-ranked predicted box as the current candidate box, and calculate the IOU value between the current candidate box and the adjacent predicted box.

[0108] (2) Based on the IOU value, the initial probability of the adjacent prediction box is reduced, and the reduced probability is used as the final probability;

[0109] (3) Remove the predicted boxes that are candidate boxes and the predicted boxes after adjusting the initial probability, and reselect the predicted box that ranks first as the current candidate box.

[0110] (4) Jump to the step of calculating the IOU value between the current candidate box and the adjacent prediction box, until the initial probability of the latest selected candidate box is less than the preset probability threshold, and obtain the final probability that each target object in the training sample image is a return component.

[0111] 205. Based on the probability and the annotation information, adjust the parameters of the SSD-Pytorch model until the SSD-Pytorch model converges to obtain the return component identification model.

[0112] In this embodiment, the probability of each target object in the training sample image being a reflow part predicted by the SSD-Pytorch model is compared with the pre-generated annotation information to determine whether the model prediction is correct. The model's performance is then evaluated using a loss function to determine whether it needs to be iterated and retrained. The specific steps are as follows:

[0113] (1) Calculate the cross-entropy loss value of the SSD-Pytorch model based on the probability and the annotation information, and determine whether the cross-entropy loss value is less than the preset loss threshold.

[0114] (2) If it is less than, the parameters of the SSD-Pytorch model are adjusted and retrained until the cross-entropy loss value is less than the preset loss threshold, and the SSD-Pytorch model is determined to have converged, thus obtaining the return part identification model.

[0115] In this implementation, the cross-entropy loss value is calculated using the Focal Loss function, which is a modified version of the traditional cross-entropy loss function. The specific cross-entropy loss function is shown below:

[0116]

[0117] Where L is the loss value, y' is the output of the activated function (between 0 and 1), and γ is a modulating factor. γ > 0 reduces the loss of easily classified samples, making the function focus more on difficult-to-classify and easily misclassified samples. For example, when γ = 2, for positive samples, the third image with a prediction result of 0.95 is a simple sample, so (1-0.95) raised to the power of γ will be very small, resulting in a smaller loss function value. However, the loss for samples with a prediction probability of 0.3 is relatively large. Similarly, for negative samples, a prediction result of 0.1 should have a much smaller loss value than a prediction result of 0.7. When the prediction probability is 0.5, the loss is reduced by only 0.25 times, so the function focuses more on these difficult-to-distinguish third images, reducing the influence of simple samples. The effect of a large number of samples with very low prediction probabilities being combined is more effective.

[0118] This invention describes in detail the training process of the return component identification model. Based on the SSD-Pytorch model, the network architecture consists of VGG Backbone layer, Extra Layers layer, and Multi-box Layers layer. Feature extraction, probability prediction, and iterative model training are performed on the training sample images. On the one hand, the Multi-box Layers layer is optimized with the Soft-NMS algorithm to reduce false positives and false negatives. On the other hand, focal loss is used to focus on training sample images that are difficult to classify and prone to misclassification.

[0119] The above describes the return component identification method in the embodiments of the present invention. The following describes the return component identification device in the embodiments of the present invention. Please refer to [link / reference]. Figure 3 One embodiment of the return component identification device in this invention includes:

[0120] The acquisition module 301 is used to acquire a first image of the detection area of ​​the return component where there is a return component and a second image of the detection area where there is no return component, and use the first image and the second image as training sample images;

[0121] The annotation module 302 is used to annotate the training sample images to obtain the annotation information of each training sample image;

[0122] Training module 303 is used to train a preset SSD-Pytorch model using the training sample images and the annotation information to obtain a return part recognition model, wherein the SSD-Pytorch model includes a VGG Backbone layer, Extra Layers layer, and Multi-box Layers layer.

[0123] The recognition module 304 is used to acquire a third image of the current return component detection area, input the third image into the return component recognition model for recognition, and output the probability that a return component exists in the third image;

[0124] The push module 305 is used to determine that there is a return component in the third image and push a prompt message if the probability is greater than a preset probability threshold.

[0125] The technical solution provided by this invention uses training sample images of the return shipment detection area and corresponding annotation information as training samples. Then, the SSD-PyTorch model is trained using these training sample images and annotation information to obtain a corresponding return shipment recognition model, which is directly applied to the subsequent recognition of return shipments. Next, when a third image of the current return shipment detection area is acquired, it is input into the return shipment recognition model for identification to predict the probability of a return shipment existing in the third image. Finally, if the predicted probability is greater than a preset probability threshold, a return shipment is identified in the third image, and a notification is sent to assist staff in determining whether a return shipment has been generated. This invention improves the accuracy of return shipment recognition, reduces the safety hazards caused by return shipments blocking the logistics line, and increases the efficiency of express delivery.

[0126] Please see Figure 4 Another embodiment of the return component identification device in this invention includes:

[0127] The acquisition module 301 is used to acquire a first image of the detection area of ​​the return component where there is a return component and a second image of the detection area where there is no return component, and use the first image and the second image as training sample images;

[0128] The annotation module 302 is used to annotate the training sample images to obtain the annotation information of each training sample image;

[0129] Training module 303 is used to train a preset SSD-Pytorch model using the training sample images and the annotation information to obtain a return part recognition model, wherein the SSD-Pytorch model includes a VGG Backbone layer, Extra Layers layer, and Multi-box Layers layer.

[0130] The recognition module 304 is used to acquire a third image of the current return component detection area, input the third image into the return component recognition model for recognition, and output the probability that a return component exists in the third image;

[0131] The push module 305 is used to determine that there is a return component in the third image and push a prompt message if the probability is greater than a preset probability threshold.

[0132] Specifically, the annotation module 302 includes:

[0133] The segmentation unit 3021 is used to delineate the region of the reflow component in the training sample image of the pre-collected reflow component detection area, and to segment the training sample image to obtain the corresponding labeled region;

[0134] The writing unit 3022 is used to write the annotation area into a preset XML format blank file to obtain the corresponding annotation information.

[0135] Specifically, the training module 303 includes:

[0136] The first extraction unit 3031 is used to input the training sample image into the VGG Backbone layer and extract the initial feature map of the training sample image through the VGG Backbone layer.

[0137] The downsampling unit 3032 is used to input the initial feature maps into the Extra Layers layer, and downsample the initial feature maps through the Extra Layers layer to obtain the corresponding feature maps;

[0138] Convolutional unit 3033 is used to input the feature map into the Multi-box Layers layer and perform convolution processing on the feature map through the Multi-box Layers layer to obtain the corresponding regression feature map and classification feature map;

[0139] The prediction unit 3034 is used to predict the probability that a return component exists in the training sample image based on the regression feature map and the classification feature map.

[0140] The adjustment unit 3035 is used to adjust the parameters of the SSD-Pytorch model according to the probability and the annotation information until the SSD-Pytorch model converges to obtain the return component identification model.

[0141] Specifically, the prediction unit 3034 includes:

[0142] Integration subunit 30341 is used to integrate the regression feature map and the classification feature map to determine the prediction boxes of multiple target objects in the training sample image, and to determine the initial probability that the target object in each prediction box is a reflow part;

[0143] The sorting subunit 30342 is used to sort the predicted bounding boxes of each target object in the training sample image in descending order of the initial probabilities;

[0144] The adjustment subunit 30343 is used to select a prediction box with an initial probability greater than a preset probability threshold as a candidate box based on the sorting order of the prediction boxes, and adjust the initial probability of the prediction boxes adjacent to the candidate boxes to obtain the final probability that each target object in the training sample image is a reflow component.

[0145] The determination subunit 30344 is used to determine the probability that a reflow component exists in the training sample image based on the final probability that each target object is a reflow component.

[0146] Specifically, the adjustment subunit 30343 is also used for:

[0147] Select the highest-ranked predicted bounding box as the current candidate bounding box, and calculate the IOU value between the current candidate bounding box and the adjacent predicted bounding boxes.

[0148] Based on the IOU value, the initial probability of the adjacent prediction box is reduced, and the reduced probability is used as the final probability;

[0149] Remove the predicted boxes that are candidate boxes and the predicted boxes after adjusting the initial probability, and reselect the predicted box that ranks first as the current candidate box.

[0150] The process jumps to the step of calculating the IOU value between the current candidate box and the adjacent predicted box, and stops when the initial probability of the latest selected candidate box is less than the preset probability threshold, thus obtaining the final probability that each target object in the training sample image is a reflow component.

[0151] Specifically, the adjustment unit 3035 includes:

[0152] The discrimination subunit 30351 is used to calculate the cross-entropy loss value of the SSD-Pytorch model based on the probability and the annotation information, and to determine whether the cross-entropy loss value is less than a preset loss threshold.

[0153] The iterative subunit 30352 is used to adjust the parameters of the SSD-Pytorch model and retrain it if the cross-entropy loss value is less than a preset loss threshold, until the cross-entropy loss value is less than the preset loss threshold, thus determining that the SSD-Pytorch model has converged and obtaining the return component identification model.

[0154] In this embodiment of the invention, an SSD-Pytorch model is constructed and trained using training sample images and annotation information to obtain a corresponding return shipment recognition model, which is directly applied to the subsequent recognition of return shipments. Then, when a third image of the current return shipment detection area is acquired, it is input into the return shipment recognition model for identification, predicting the probability that each logistics package in the third image is a return shipment. The invention also details how the SSD-Pytorch model is used as the base model, with VGG Backbone layers, Extra Layers, and Multi-box Layers as the network architecture. Feature extraction, probability prediction, and iterative model training are performed on the training sample images. On one hand, the Multi-box Layers are optimized using the Soft-NMS algorithm to reduce false positives and false negatives. On the other hand, focal loss is used to focus on training sample images that are difficult to classify and prone to misclassification, improving the recognition accuracy of return shipments, reducing safety hazards caused by return shipments blocking the logistics line, and increasing the efficiency of express delivery.

[0155] above Figure 3 and Figure 4 The return component identification device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The return component identification device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0156] Figure 5 This is a schematic diagram of a reflow device identification device 500 provided in an embodiment of the present invention. The reflow device identification device 500 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the reflow device identification device 500. Furthermore, the processor 510 may be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the reflow device identification device 500.

[0157] The return item identification device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 5 The illustrated structure of the return component identification device does not constitute a limitation on the return component identification device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0158] The present invention also provides a return component identification device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the return component identification method in the above embodiments.

[0159] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the reflow identification method.

[0160] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0161] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A backflow member identification method characterized by comprising: The backflow member identification method comprises: acquiring a first image of a backflow member detection area with a backflow member and a second image of the backflow member detection area without the backflow member, and taking the first image and the second image as training sample images; inputting the training sample images into preset image labeling software for display, and circumscribing the area of the backflow member in the training sample images to generate labeling information; training a preset SSD-Pytorch model using the training sample images and the labeling information to obtain a backflow member identification model, wherein the SSD-Pytorch model comprises a VGG Backbone layer, an Extra Layers layer, and a Multi-box Layers layer; acquiring a third image of a current backflow member detection area, and inputting the third image into the backflow member identification model for identification to output the probability of the presence of a backflow member in the third image; if the probability is greater than a preset probability threshold, it is determined that the third image contains a backflow member and a prompt information is pushed; the training of the preset SSD-Pytorch model using the training sample images and the labeling information to obtain a backflow member identification model comprises: inputting the training sample images into the VGG Backbone layer and extracting initial feature maps of the training sample images through the VGG Backbone layer; inputting the initial feature maps into the Extra Layers layer and performing down-sampling on the initial feature maps through the Extra Layers layer to obtain corresponding feature maps; inputting the feature maps into the Multi-box Layers layer and performing convolution processing on the feature maps through the Multi-box Layers layer to obtain corresponding regression feature maps and classification feature maps; integrating the regression feature maps and the classification feature maps, determining the prediction boxes of multiple target objects in the training sample images, and determining the initial probability of the target objects in the prediction boxes being backflow members; sorting the prediction boxes of the target objects in the training sample images in descending order of the initial probability; based on the sorting order of the prediction boxes, selecting the prediction boxes with an initial probability greater than a preset probability threshold as candidate boxes, and adjusting the initial probability of the prediction boxes adjacent to the candidate boxes to obtain the final probability of the target objects in the training sample images being backflow members; based on the final probability of the target objects being backflow members, determining the probability of the presence of a backflow member in the training sample images; adjusting the parameters of the SSD-Pytorch model according to the probability and the labeling information until the SSD-Pytorch model converges to obtain a backflow member identification model.

2. The method according to claim 1, wherein the inputting of the training sample images into the preset image labeling software for display and the circumscription of the area of the backflow member in the training sample images to generate labeling information comprises: Circle the area range of the backflow component in the training sample image of the pre-acquired backflow component detection area, and segment the training sample image to obtain the corresponding labeled area; Write the labeled area into a preset blank file in XML format to obtain corresponding labeling information.

3. The method of claim 1, wherein The final probability of each target object being a backflow component in the training sample image includes: Select the top-ranked prediction box as the current candidate box and calculate the IOU value of the current candidate box and the adjacent prediction box; Based on the IOU value, the initial probability of the adjacent prediction box is adjusted, and the adjusted probability is taken as the final probability; Remove the prediction box as the candidate box and the prediction box after adjusting the initial probability, and select the top-ranked prediction box as the current candidate box again; Jump to the step of calculating the IOU value of the current candidate box and the adjacent prediction box until the initial probability of the newly selected candidate box is less than the preset probability threshold, and the final probability of each target object being a backflow component in the training sample image is obtained.

4. The method of claim 1, wherein The parameters of the SSD-Pytorch model are adjusted according to the probability and the labeling information until the SSD-Pytorch model converges, and a backflow component recognition model is obtained, including: According to the probability and the labeling information, the cross-entropy loss value of the SSD-Pytorch model is calculated, and it is judged whether the cross-entropy loss value is less than the preset loss threshold; If it is less than, the parameters of the SSD-Pytorch model are adjusted and retrained until the cross-entropy loss value is less than the preset loss threshold, the SSD-Pytorch model is determined to be converged, and a backflow component recognition model is obtained.

5. A backflow member identification device characterized by comprising: The backflow component recognition device includes: An acquisition module is configured to acquire a first image with a backflow component and a second image without a backflow component in a pre-acquired backflow component detection area, and take the first image and the second image as training sample images; A labeling module is configured to input the training sample images into a preset image labeling software for display, and to circle the area of the backflow component in the training sample images to generate labeling information; A training module is configured to train a preset SSD-Pytorch model using the training sample images and the labeling information to obtain a backflow component recognition model, wherein the SSD-Pytorch model includes a VGG Backbone layer, an ExtraLayers layer, and a Multi-box Layers layer; An identification module is configured to acquire a third image of a current backflow component detection area, and input the third image into the backflow component recognition model for identification to output a probability of the third image having a backflow component; A pushing module is configured to determine that the third image has a backflow component and push a prompt information if the probability is greater than a preset probability threshold. The training sample image and the labeling information are used to train a preset SSD-Pytorch model to obtain a reflux component recognition model. The training sample image is input into the VGG Backbone layer, and initial feature maps of the training sample image are extracted through the VGG Backbone layer. Each initial feature map is input into the Extra Layers layer, and the initial feature map is down-sampled through the Extra Layers layer to obtain a corresponding feature map. The feature map is input into the Multi-box Layers layer, and the feature map is convoluted through the Multi-box Layers layer to obtain a corresponding regression feature map and a classification feature map. The regression feature map and the classification feature map are integrated, the prediction boxes of multiple target objects in the training sample image are determined respectively, and the initial probability of the target object in each prediction box being a reflux component is determined. The initial probabilities are sorted in descending order to sort the prediction boxes of each target object in the training sample image. Based on the sorting order of the prediction boxes, a prediction box with an initial probability greater than a preset probability threshold is selected as a candidate box, and the initial probability of a prediction box adjacent to the candidate box is adjusted to obtain the final probability of the target object in the training sample image being a reflux component. Based on the final probability of each target object being a reflux component, the probability of the training sample image containing a reflux component is determined. The parameters of the SSD-Pytorch model are adjusted according to the probability and the labeling information until the SSD-Pytorch model converges, and a reflux component recognition model is obtained.

6. The apparatus of claim 5, wherein The labeling module comprises: A segmentation unit is configured to circumscribe the area range of the reflux component in the training sample image pre-acquired from the reflux component detection area, and segment the training sample image to obtain a corresponding labeled area. A writing unit is configured to write the labeled area into a preset blank file in XML format to obtain corresponding labeling information.

7. A reflow member identification apparatus characterized by comprising: The reflux component recognition device comprises a memory and at least one processor, and the memory stores instructions. The at least one processor invokes the instructions in the memory to enable the reflux component recognition device to perform the reflux component recognition method of any one of claims 1-4.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the reflux component recognition method of any one of claims 1-4.

Images