Multi-parcel adhesion identification method, device and equipment for logistics conveying belt and storage medium

Abstract

The invention relates to the field of logistics, and discloses a logistics conveyor belt multi-parcel adhesion identification method, device and equipment and a storage medium, and the method is used for detecting whether adhesion parcels exist on a logistics conveyor belt or not. The method comprises the following steps: constructing an improved PSPNet model based on a space attention mechanism, and training to obtain an adhesive parcel recognition model; obtaining a package RGB image, a package depth image and a package infrared thermogram of the logistics conveyor belt, respectively extracting texture features of the package RGB image, geometric features of the package depth image and temperature features of the package infrared thermogram, and fusing the texture features, the geometric features and the temperature features to obtain fusion features; the fusion features are input into an adhesive parcel recognition model, a detection result is obtained, and the detection result comprises bounding box coordinates, segmentation masks and confidence coefficients; and if the confidence exceeds a preset confidence threshold, extracting the specific position and segmentation information of the adhered parcel from the bounding box coordinate and the segmentation mask, and generating early warning information.

Description

Technical Field

[0001] This invention relates to the field of logistics, and in particular to a method, apparatus, equipment, and storage medium for identifying multiple packages stuck together on a logistics conveyor belt. Background Technology

[0002] With the booming development of e-commerce, the parcel sorting volume in the logistics industry has increased exponentially, making automated sorting systems a core component for improving efficiency. Among these, the identification of multiple parcels adhering to each other in conveyor belt scenarios is a key technological bottleneck in automated sorting. When multiple parcels overlap due to stacking, squeezing, or friction, traditional sorting equipment is prone to misjudging them as a single parcel, leading to sorting errors, reduced efficiency, and even parcel damage.

[0003] Therefore, accurately identifying stuck packages and locating their boundaries is of great significance for achieving intelligent and efficient logistics sorting.

[0004] Currently, the technology for identifying sticky packages in logistics mainly relies on traditional computer vision methods and single-modal deep learning methods, but both have significant shortcomings.

[0005] Early traditional computer vision methods often employed texture analysis based on RGB images (such as HOG and SIFT features) or edge detection (such as the Canny operator) to distinguish packages by differences in color and shape. However, these methods are significantly affected by changes in lighting and the similarity of package appearances. When packages have the same color and uniform texture (such as all being brown cardboard boxes), the boundaries of adhered areas are difficult to define, resulting in an accuracy rate of less than 60%.

[0006] In recent years, deep learning-based semantic segmentation models (such as the original PSPNet and U-Net) have been applied to package segmentation, but most of them rely on a single RGB image input. Although they have improved the segmentation accuracy of single packages, when faced with stacked packages, partially occluded or other adhered scenarios, the lack of spatial geometric or physical attribute information makes it easy to misjudge adhered areas as a whole, resulting in a recall rate of less than 70% for adhered package recognition.

[0007] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0008] This invention provides a method, apparatus, device, and storage medium for identifying multiple packages stuck together on a logistics conveyor belt, used to detect whether there are stuck packages on a logistics conveyor belt.

[0009] The first aspect of this invention provides a method for identifying multiple packages stuck together on a logistics conveyor belt. The method includes: constructing an improved PSPNet model based on a spatial attention mechanism and training the improved PSPNet model to obtain a package sticking together identification model; acquiring RGB images, depth images, and infrared thermal images of packages from the logistics conveyor belt, and extracting texture features from the RGB images, geometric features from the depth images, and temperature features from the infrared thermal images; fusing the texture features, geometric features, and temperature features to obtain fused features; inputting the fused features into the package sticking together identification model and obtaining the detection results output by the model, the detection results including bounding box coordinates, a segmentation mask, and a confidence score; if the confidence score exceeds a preset confidence threshold, extracting the specific location and segmentation information of the stuck packages from the bounding box coordinates and the segmentation mask, and generating warning information based on the specific location and segmentation information of the stuck packages.

[0010] Optionally, in a first implementation of the first aspect of the present invention, the improved PSPNet model is constructed based on the spatial attention mechanism, and the improved PSPNet model is trained to obtain an adhesive package recognition model; the improved PSPNet model is constructed based on the spatial attention mechanism; historical multimodal data of the logistics conveyor belt is acquired, and a training set and a test set are constructed based on the historical multimodal data; based on the training set, the improved PSPNet model is trained using the ten-fold cross-validation method to obtain a trained PSPNet model, and the trained PSPNet model is tested using the test set; when the test result is qualified, the trained PSPNet model is used as the adhesive package recognition model.

[0011] Optionally, in a second implementation of the first aspect of the present invention, the construction of the improved PSPNet model based on the spatial attention mechanism includes: constructing a feature extraction layer, a pyramid pooling module, and an output layer of the PSPNet model, wherein the pyramid pooling module adopts a progressively dilated convolutional pyramid; constructing a spatial attention module and embedding the spatial attention module between the feature extraction layer and the pyramid pooling module to obtain the improved PSPNet model; the feature extraction layer is used to form a fused feature map based on the fused features; the spatial attention module is used to receive the fused feature map, perform weighted processing on the fused feature map, and output a weighted feature map; the progressively dilated convolutional pyramid is used to perform multi-scale fusion on the weighted feature map and output a multi-scale fused feature map to the output layer; the output layer processes the multi-scale fused feature map and outputs the bounding box coordinates, segmentation mask, and confidence score.

[0012] Optionally, in a third implementation of the first aspect of the present invention, the step of acquiring the RGB image, depth image, and infrared thermal image of the package from the logistics conveyor belt, and extracting the texture features of the RGB image, the geometric features of the depth image, and the temperature features of the infrared thermal image, respectively, includes: acquiring the RGB image, depth image, and infrared thermal image of the package from the logistics conveyor belt; extracting the region of interest from the RGB image and extracting the texture features of the RGB image from the extracted region of interest; converting the depth image into point cloud data and extracting the geometric features of the depth image based on the point cloud data; converting the grayscale value of the infrared thermal image into a temperature value and extracting the temperature features of the infrared thermal image based on the temperature value.

[0013] Optionally, in a fourth implementation of the first aspect of the present invention, fusing the texture features, the geometric features, and the temperature features to obtain a fused feature includes: mapping the texture features, the geometric features, and the temperature features to a 2048 embedding space to obtain mapped texture features, mapped geometric features, and mapped temperature features; calculating the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features respectively; and fusing the mapped texture features, the mapped geometric features, and the mapped temperature features based on the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features to obtain the fused feature.

[0014] Optionally, in a fifth implementation of the first aspect of the present invention, the step of inputting the fused features into the adhesive package recognition model and obtaining the detection result output by the adhesive package recognition model, wherein the detection result includes bounding box coordinates, segmentation mask, and confidence score, includes: reshaping the fused features into a feature map format adapted to the adhesive package recognition model; inputting the reshaped fused features into the adhesive package recognition model and obtaining the detection result output by the adhesive package recognition model, wherein the detection result includes bounding box coordinates, segmentation mask, and confidence score.

[0015] Optionally, in a sixth implementation of the first aspect of the present invention, the step of extracting the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold, and generating warning information based on the specific location and segmentation information of the stuck package, includes: extracting the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold; determining the warning level based on the confidence level; and generating warning information based on the warning level, the specific location and segmentation information of the stuck package.

[0016] A second aspect of the present invention provides a multi-package adhesion recognition device for logistics conveyors, comprising: a training module for constructing an improved PSPNet model based on a spatial attention mechanism and training the improved PSPNet model to obtain an adhesion package recognition model; an extraction module for acquiring RGB images, depth images, and infrared thermal images of packages on the logistics conveyor belt, and extracting texture features from the RGB images, geometric features from the depth images, and temperature features from the infrared thermal images, respectively; a fusion module for fusing the texture features, geometric features, and temperature features to obtain fused features; a recognition module for inputting the fused features into the adhesion package recognition model and obtaining the detection results output by the adhesion package recognition model, the detection results including bounding box coordinates, segmentation masks, and confidence scores; and a generation module for extracting the specific location and segmentation information of the adhesion packages from the bounding box coordinates and the segmentation mask if the confidence score exceeds a preset confidence threshold, and generating warning information based on the specific location and segmentation information of the adhesion packages.

[0017] Optionally, in a first implementation of the second aspect of the present invention, the training module includes: a construction unit for constructing an improved PSPNet model based on a spatial attention mechanism; an acquisition unit for acquiring historical multimodal data of a logistics conveyor belt and constructing a training set and a test set based on the historical multimodal data; and a training unit for training the improved PSPNet model using ten-fold cross-validation based on the training set to obtain a trained PSPNet model, and testing the trained PSPNet model using the test set. When the test result is qualified, the trained PSPNet model is used as a model for recognizing sticky packages.

[0018] Optionally, in a second implementation of the second aspect of the present invention, the extraction module includes: a first extraction unit, configured to acquire an RGB image of a package, a depth image of a package, and an infrared thermal image of a package from a logistics conveyor belt; extract a region of interest from the RGB image of the package; and extract texture features of the RGB image of the package from the extracted region of interest; a second extraction unit, configured to convert the depth image of the package into point cloud data; and extract geometric features of the depth image of the package based on the point cloud data; and a third extraction unit, configured to convert the grayscale values ​​of the infrared thermal image of the package into temperature values; and extract temperature features of the infrared thermal image of the package based on the temperature values.

[0019] Optionally, in a third implementation of the second aspect of the present invention, the fusion module includes: a mapping unit, configured to map the texture features, the geometric features, and the temperature features to a 2048 embedding space to obtain mapped texture features, mapped geometric features, and mapped temperature features; a calculation unit, configured to calculate the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features respectively; and a fusion unit, configured to fuse the mapped texture features, the mapped geometric features, and the mapped temperature features based on the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features to obtain fused features.

[0020] Optionally, in a fourth implementation of the second aspect of the present invention, the recognition module includes: a reshaping unit, used to reshape the fused features into a feature map format adapted to the adhesive package recognition model; and a recognition unit, used to input the reshaped fused features into the adhesive package recognition model and obtain the detection results output by the adhesive package recognition model, wherein the detection results include bounding box coordinates, segmentation mask, and confidence level.

[0021] Optionally, in a fifth implementation of the second aspect of the present invention, the generation module includes: a fourth extraction unit, configured to extract the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold; a judgment unit, configured to determine the warning level based on the confidence level; and a generation unit, configured to generate warning information based on the warning level, the specific location and segmentation information of the stuck package.

[0022] A third aspect of the present invention provides a multi-package adhesion identification device for a logistics conveyor belt, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the computer-readable instructions in the memory to cause the multi-package adhesion identification device for the logistics conveyor belt to perform the various steps of the multi-package adhesion identification method for the logistics conveyor belt as described above.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the computer to perform the steps of the multi-package adhesion identification method for logistics conveyor belts as described above.

[0024] The technical solution provided by this invention, by introducing the PSPNet model with a spatial attention mechanism, can specifically focus on the key spatial features of the package adhesion area, strengthen the feature extraction of sensitive areas such as package edges and contact boundaries, effectively suppress the interference of irrelevant information such as conveyor belt background and noise, and improve the model's feature discrimination ability in complex adhesion scenarios. Moreover, the acquisition and feature extraction of multimodal images (RGB, depth, infrared thermal images) characterize package attributes from multiple dimensions such as texture, geometry, and temperature, avoiding the limitations of a single data source. The feature fusion strategy further integrates the advantages of each modality, further improving the model's adaptability to complex adhesion scenarios. In addition, by combining confidence judgment and accurate location information extraction, timely and effective early warnings can be generated, providing reliable technical support for the automation of logistics sorting, significantly reducing the cost of manual intervention, and improving sorting efficiency and accuracy. Attached Figure Description

[0025] Figure 1 This is a first flowchart of a method for identifying multiple packages adhering to a logistics conveyor belt, provided in an embodiment of the present invention.

[0026] Figure 2 This is a second flowchart of the multi-package adhesion identification method for logistics conveyor belts provided in an embodiment of the present invention;

[0027] Figure 3 This is a third flowchart of the multi-package adhesion identification method for logistics conveyor belts provided in this embodiment of the invention;

[0028] Figure 4 This is a fourth flowchart of the multi-package adhesion identification method for logistics conveyor belts provided in this embodiment of the invention;

[0029] Figure 5 This is a fifth flowchart of the multi-package adhesion identification method for logistics conveyor belts provided in an embodiment of the present invention;

[0030] Figure 6 The sixth flowchart of the multi-package adhesion identification method for logistics conveyor belts provided in the embodiments of the present invention;

[0031] Figure 7 This is a schematic diagram of the structure of the multi-package adhesion identification device for logistics conveyor belts provided in an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the structure of a multi-package adhesion identification device for a logistics conveyor belt provided in an embodiment of the present invention. Detailed Implementation

[0033] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of a method for identifying multiple packages adhering to a logistics conveyor belt according to the present invention includes:

[0035] S101. An improved PSPNet model is constructed based on the spatial attention mechanism, and the improved PSPNet model is trained to obtain an adhesive package recognition model.

[0036] In this embodiment, a model for recognizing adhered packages is trained by embedding a spatial attention module into the backbone network of PSPNet as the basic framework.

[0037] The spatial attention mechanism achieves a precise transition from "global feature extraction" to "local key area enhancement" through dynamic allocation of spatial weights. It not only adapts to the spatial distribution characteristics of package adhesion in logistics scenarios, but also leverages the complementary advantages of multimodal features to provide high-quality feature inputs for subsequent multi-scale fusion and detection output, ultimately improving the accuracy and robustness of the adhered package recognition model.

[0038] It is understood that the executing entity of this invention can be a multi-package adhesion identification device for logistics conveyor belts, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0039] S102. Obtain the RGB image, depth image and infrared thermal image of the package from the logistics conveyor belt, and extract the texture features of the RGB image, the geometric features of the depth image and the temperature features of the infrared thermal image of the package respectively.

[0040] In this embodiment, an industrial CCD camera is used, mounted directly above or to the side of the conveyor belt, to capture a color image of the package's appearance, resulting in an RGB image of the package. The frame rate matches the conveyor belt speed (typically 10-30 fps).

[0041] Using a ToF camera or a structured light depth sensor, the three-dimensional depth information of the package is acquired to obtain a package depth image, which reflects the spatial features of the package, such as its height and stacking thickness.

[0042] Infrared thermal imagers are used to capture the temperature distribution on the surface of packages to obtain infrared thermal images of the packages. Packages made of different materials will exhibit different temperature characteristics due to differences in their thermal conductivity.

[0043] S103. Fuse texture features, geometric features, and temperature features to obtain fused features;

[0044] In this embodiment, the RGB texture features (2048-dimensional), depth geometric features (512-dimensional), and infrared temperature features (512-dimensional) are first aligned in dimensions, and a fully connected layer is used to map the depth and temperature features to 2048 dimensions.

[0045] Then, the attention weights of the three features are obtained (the sum of the weights is 1), and the fused feature vector (2048 dimensions) is obtained by weighted summation.

[0046] S104. Input the fused features into the sticky package recognition model and obtain the detection results output by the sticky package recognition model. The detection results include bounding box coordinates, segmentation mask and confidence score.

[0047] In this embodiment, the output layer of the adhesive package recognition model is divided into two branches: a bounding box branch and a segmentation mask branch.

[0048] The bounding box branch predicts the coordinates (x1, y1, x2, y2) of the top left and bottom right corners of each package, reflecting the range of the package's position in the image.

[0049] The segmentation mask branch outputs a mask image of the same size as the input image. Regions with a mask value of 1 represent enclosed pixels, while 0 represents the background, achieving pixel-level contour segmentation.

[0050] The adhesive package recognition model also outputs a value between 0 and 1, i.e., the confidence level, based on the degree of matching between the prediction results and the actual labels. The closer the confidence level is to 1, the more reliable the prediction is.

[0051] S105. If the confidence level exceeds the preset confidence threshold, the specific location and segmentation information of the stuck packages are extracted from the bounding box coordinates and segmentation mask, and an early warning message is generated based on the specific location and segmentation information of the stuck packages.

[0052] In this embodiment, a confidence threshold is preset (usually set to 0.7, which can be adjusted according to the actual scenario). When the confidence level of the model output exceeds the preset confidence threshold, it is determined to be a valid identification result, which means that there is an adhesive package.

[0053] Extract the center position ((x1+x2) / 2,(y1+y2) / 2) and coverage area of ​​the stuck wrapper from the bounding box coordinates; extract the pixel percentage of the stuck region (reflecting the degree of sticking) from the segmentation mask.

[0054] The warning information includes the number of stuck packages, their location coordinates (corresponding to the actual physical location of the conveyor belt), and the degree of stickiness (slight / severe). The warning can be triggered by pop-up windows on the system interface, audible and visual alarms, or by connecting to the sorting control system, to initiate response actions such as pausing sorting or adjusting the robotic arm.

[0055] This embodiment provides a method for identifying multiple packages stuck together on a logistics conveyor belt. By introducing a PSPNet model with a spatial attention mechanism, it can specifically focus on the key spatial features of the package sticking area, strengthen feature extraction of sensitive areas such as package edges and contact boundaries, effectively suppress interference from irrelevant information such as conveyor belt background and noise, and improve the model's feature discrimination ability in complex sticking scenarios. Moreover, the acquisition and feature extraction of multimodal images (RGB, depth, infrared thermal images) characterize package attributes from multiple dimensions such as texture, geometry, and temperature, avoiding the limitations of a single data source. The feature fusion strategy further integrates the advantages of each modality, further improving the model's adaptability to complex sticking scenarios. In addition, by combining confidence judgment and accurate location information extraction, timely and effective early warnings can be generated, providing reliable technical support for the automation of logistics sorting, significantly reducing the cost of manual intervention, and improving sorting efficiency and accuracy.

[0056] Please see Figure 2 The second embodiment of the multi-package adhesion identification method for logistics conveyor belts in this invention includes:

[0057] S201. An improved PSPNet model is constructed based on the spatial attention mechanism;

[0058] In this embodiment, an improved PSPNet model is constructed based on a spatial attention mechanism, including: constructing a feature extraction layer, a pyramid pooling module, and an output layer of the PSPNet model. The pyramid pooling module adopts a progressively dilated convolutional pyramid. A spatial attention module is constructed and embedded between the feature extraction layer and the pyramid pooling module to obtain the improved PSPNet model. The feature extraction layer is used to form a fused feature map based on the fused features. The spatial attention module is used to receive the fused feature map, perform weighted processing on the fused feature map, and output a weighted feature map. The progressively dilated convolutional pyramid is used to perform multi-scale fusion on the weighted feature map and output a multi-scale fused feature map to the output layer. After processing the multi-scale fused feature map, the output layer outputs the bounding box coordinates, segmentation mask, and confidence score.

[0059] In this embodiment, the feature extraction layer uses ResNet-101 as the backbone network. Through multiple sets of convolutional operations (such as 7×7 convolution and 3×3 convolution) and residual connections, the input fused features are transformed into a high-dimensional initial feature map (the dimension is usually 2048×H×W), realizing the layer-by-layer extraction of image features from low-level features (edges, textures) to high-level features (semantic information).

[0060] In this embodiment, a spatial attention module is inserted between the last convolutional layer of ResNet-101 and the pyramid pooling module.

[0061] The spatial attention module consists of an input layer, a core processing layer, and an output layer.

[0062] The input layer receives the fused feature map output by the feature extraction layer. The fusion feature map has dimensions of C×H×W (C is the number of channels, and H×W is the spatial size) and contains multimodal information such as RGB texture, depth geometry, and infrared temperature.

[0063] Function: To provide the original feature basis for subsequent spatial weight calculation, ensuring that the weight allocation can cover the spatial distribution characteristics of each mode (such as temperature change points, depth edges, etc.).

[0064] The core processing layer generates spatial attention weights through two feature transformations, specifically including:

[0065] ① Feature aggregation unit: The feature aggregation unit performs two pooling operations on the input feature map: global average pooling and global max pooling. Then, the two pooling results are concatenated according to the channel dimension to obtain a 2×H×W aggregated feature map, which integrates global statistics and local saliency information.

[0066] Global average pooling is used to calculate the channel mean at each spatial location, generating a 1×H×W feature map (reflecting the overall feature intensity of the region);

[0067] Global max pooling is used to calculate the maximum channel value at each spatial location, generating a 1×H×W feature map (reflecting local salient features of the region, such as gradient peaks at adhesion boundaries).

[0068] ② Convolution and activation units: The convolution and activation units use 3×3 convolutional layers (stride = 1, padding = 1) to process the aggregated feature map, keeping the spatial size unchanged (still H×W), while compressing the number of channels to 1; after passing through the Sigmoid activation function, a spatial weight map α∈[0,1]1×H×W is generated, where the closer the value is to 1, the more important the spatial location is for adhesion recognition (such as the edge of the package contact, the area of ​​material difference).

[0069] The output layer multiplies the original fused feature map with the spatial weight map pixel by pixel, and outputs a weighted feature map with dimensions C×H×W.

[0070] The spatial attention module is used to enhance the feature response of high-value regions in the weight map (such as texture, geometry, and temperature signals of adhesion boundaries) and suppress low-value regions (such as conveyor belt background and irrelevant noise). The output weighted feature map is directly input into the subsequent progressively dilated convolutional pyramid for multi-scale fusion.

[0071] In this embodiment, the progressively dilated convolutional pyramid improves the accuracy of multi-scale feature fusion. The progressively dilated convolutional pyramid comprises four parallel branches, employing dilated convolutional layers with dilation rates of 1, 2, 4, and 8 respectively: the branch with dilation rate 1 focuses on local details of the package (such as surface texture and small package edges) through uninterrupted convolution; the branch with dilation rate 2 captures medium-scale features (such as the main outline of the package); and the branches with dilation rates 4 and 8 expand the receptive field to capture larger-scale package distribution (such as the relative positions of multiple packages) and global scene information (such as the overall layout of the conveyor belt), respectively. The feature maps output from each branch are upsampled to the same scale and then concatenated to form aggregated features containing multi-scale information from local to global perspectives. Compared to traditional pooling, this method more efficiently preserves feature resolution and is suitable for the large differences in package scale (from envelopes to large cardboard boxes) in logistics scenarios.

[0072] S202. Obtain historical multimodal data of the logistics conveyor belt, and construct training and testing sets based on the historical multimodal data.

[0073] In this embodiment, multimodal data from the logistics conveyor belt over the past 6-12 months are collected as historical multimodal data, and the historical multimodal data is cleaned and labeled.

[0074] Then, the processed data is divided into training and test sets in an 8:2 ratio. Stratified sampling is used during the division to ensure that the distribution of package type, adhesion degree and environmental conditions is consistent between the training and test sets (e.g., the proportion of "cardboard box-cloth bag adhesion" samples in the training set deviates from that in the test set by no more than 5%), so as to avoid distortion of test results due to uneven data distribution.

[0075] S203. Based on the training set, the improved PSPNet model is trained using the ten-fold cross-validation method to obtain the trained PSPNet model. The trained PSPNet model is then tested using the test set. When the test result is qualified, the trained PSPNet model is used as the adhesive package recognition model.

[0076] In this embodiment, the training set is randomly divided into 10 folds, and the sample distribution of each fold is consistent with that of the entire training set (achieved through stratified sampling).

[0077] During training, nine subsets were selected in turn as training data and one subset as validation data, for a total of 10 rounds of training.

[0078] In each training round, the optimizer uses AdamW (Adam with weight decay), with an initial learning rate of 0.0001, which decays to 0.9 times the previous round every 3 epochs. The loss function uses a hybrid loss, combining semantic segmentation loss (cross-entropy loss), bounding box regression loss (Smooth L1 loss), and sticky label classification loss (Focal Loss), with a weight ratio of 1:0.5:0.3.

[0079] Each training session consists of 30 epochs, during which the mean intersection-union ratio (mIoU), bounding box accuracy (proportion of IoU ≥ 0.5), and adhesion classification accuracy are recorded on the validation set.

[0080] After 10 rounds of training, the average performance index of the 10 models is calculated, and the model with the best performance is selected as the candidate model.

[0081] Finally, the candidate models were evaluated using a test set. The key metrics included: mIoU of the segmentation mask (≥0.85), average accuracy of bounding box detection (AP50, accuracy when IoU ≥0.5, ≥0.90), and F1 score of adhesion recognition (≥0.88).

[0082] If all indicators exceed the preset thresholds (as mentioned above) and the performance is stable in different scene subsets (such as low light and high stacking scenes) (indicator fluctuation ≤ 5%), the test result is considered qualified, and the model is the final adhesive package recognition model; if it is unqualified, it needs to return to the training stage, adjust the model structure or expand the targeted data for retraining.

[0083] In this embodiment, by fusing the spatial attention module with PSPNet, the model can capture both the spatial scale features of the package (such as size and stacking shape) and the dynamic changes in the time dimension (such as positional shift during conveyor belt movement and evolution of adhesion state). This breaks through the limitations of traditional static models in adapting to dynamic scenes, improves the recognition accuracy of the adhesive package recognition model, and achieves high-precision recognition of adhesive packages.

[0084] Please see Figure 3 A third embodiment of a method for identifying multiple packages adhering to a logistics conveyor belt, as described in this invention, includes:

[0085] S301. Obtain the RGB image, depth image, and infrared thermal image of the package from the logistics conveyor belt. Extract the region of interest from the RGB image of the package and extract the texture features of the RGB image of the package from the extracted region of interest.

[0086] In this embodiment, a 20-megapixel industrial CCD camera (30fps, adjustable exposure time) is used, installed 1.5 meters directly above the conveyor belt to ensure full coverage of the conveyor belt width (typically 1-2 meters). The industrial CCD camera captures RGB images of the packages on the logistics conveyor belt.

[0087] In this embodiment, the enhanced image is binarized using the Otsu thresholding method (to distinguish the package from the conveyor belt background), the package outline is extracted using Canny edge detection, and then the internal holes of the outline are filled using morphological closing operations (5×5 rectangular structuring elements). Finally, the region of interest containing only the package area is obtained, and background pixels are removed (the proportion is usually reduced to less than 10%).

[0088] Local binary pattern (LBP) histograms (radius 3, neighborhood 8 points) are calculated within the region of interest to describe the micro-texture. Gray-level co-occurrence matrix (GLCM, distance 1, angle 0° / 45° / 90° / 135°) is calculated to extract statistical features such as energy, entropy, and contrast, resulting in a total of 128-dimensional feature vectors.

[0089] In this embodiment, a structured light depth camera is used to acquire the depth image of packages on the logistics conveyor belt. The structured light depth camera has an accuracy of ±2mm and a resolution of 640×480. The structured light depth camera and the industrial CCD camera are triggered synchronously (time error <1ms). The installation position is coaxial with the industrial CCD camera to ensure that the field of view overlap is >95%. The depth value (unit: mm) of each pixel is acquired.

[0090] In this embodiment, an infrared thermal imager with a resolution of 640×512 is used to collect infrared thermal images of packages on the logistics conveyor belt. The temperature measurement range of the infrared thermal imager is -20℃ to 150℃, and the accuracy is ±2%. The infrared thermal imager is installed next to the industrial CCD camera (2 meters away from the conveyor belt), and the lens angle is adjusted to coincide with the field of view of the industrial CCD camera.

[0091] S302. Convert the package depth image into point cloud data, and extract the geometric features of the package depth image based on the point cloud data.

[0092] In this embodiment, the package depth image is converted into point cloud data. After conversion, outliers are removed by statistical filtering, that is, points within the range of the mean ± 3 standard deviations are retained to obtain a clean package point cloud.

[0093] In this embodiment, the DBSCAN algorithm is used to cluster the point cloud of each package to distinguish different packages; and the geometric features of a single package are calculated, including spatial volume, surface area, center coordinates, and length, width and height.

[0094] In this embodiment, the spatial volume enclosed by the point cloud is estimated by the convex hull algorithm. After triangulating the point cloud, the total area of ​​the triangles and the surface area are calculated. The center coordinates are the three-dimensional coordinates of the centroid of the point cloud, and the maximum distances of the length, width, and height along the X, Y, and Z axes are calculated.

[0095] In this embodiment, the geometric relationship features of adjacent packages are calculated, including center distance, height difference and projection overlap rate. The center distance is the Euclidean distance between the centroids of the two packages, the height difference is the Z-axis coordinate difference, and the projection overlap rate is the proportion of overlapping projected areas in the XY plane.

[0096] S303. Convert the grayscale values ​​of the package infrared thermal image into temperature values, and extract the temperature features of the package infrared thermal image based on the temperature values.

[0097] In this embodiment, the temperature region of a single package is mapped to an infrared thermal image using point cloud clustering results to obtain corresponding pixels. Basic statistical features are then calculated, including temperature mean, variance, maximum / minimum value, and temperature difference. The temperature mean reflects the overall temperature level, the variance reflects the uniformity of temperature distribution, the maximum / minimum value can capture local hot or cold spots, and the temperature difference is the difference between the maximum and minimum values.

[0098] In this embodiment, extracting temperature distribution features includes drawing a temperature histogram, calculating the peak position (most likely temperature) and peak width (degree of temperature concentration) of the histogram; calculating the temperature gradient through a sliding window (3×3), and extracting the maximum gradient value (reflecting the area of ​​temperature abrupt change, usually the edge of the wrapping or the splicing of materials).

[0099] In this embodiment, by simultaneously acquiring RGB, depth, and infrared images, a multi-dimensional feature system combining visual texture, spatial geometry, and temperature characteristics is constructed. This overcomes the information limitations of a single image source in complex scenes. RGB images capture differences in the appearance of the package, depth images restore three-dimensional spatial relationships, and infrared thermal images reveal differences in material thermal conductivity. The three complement each other and can cover the recognition needs of different lighting, materials, and adhesion patterns. At the same time, targeted preprocessing steps (ROI extraction, point cloud conversion, and grayscale conversion) effectively filter out noise and redundant information, allowing feature extraction to focus on the core area of ​​the package and avoid background interference. Furthermore, the feature extraction method is deeply adapted to the modal characteristics, ensuring that the extracted features have strong discriminative power and providing high-quality data for subsequent fusion and recognition.

[0100] Please see Figure 4 A fourth embodiment of a method for identifying multiple packages adhering to each other on a logistics conveyor belt, as described in this invention, includes:

[0101] S401. Map the texture features, geometric features, and temperature features to the 2048 embedding space to obtain the mapped texture features, mapped geometric features, and mapped temperature features.

[0102] In this embodiment, the texture features are 4224-dimensional, containing rich details of the package's appearance; the geometric features are 18-dimensional, reflecting the spatial shape of the package; and the temperature features are 32-dimensional, reflecting the temperature differences of the material. Mapping the texture features, geometric features, and temperature features to a 2048-dimensional space can reduce the information loss of high-dimensional features during compression.

[0103] Using a fully connected mapping network adapted to each feature, texture features, geometric features, and temperature features are mapped to a 2048 embedding space.

[0104] The texture feature mapping network uses three fully connected layers. The first layer compresses 4224 dimensions to 3072 dimensions (activation function ReLU), the second layer compresses to 2560 dimensions (activation function ReLU), and the third layer compresses to 2048 dimensions (activation function Tanh). Each layer adds a Batch Normalization layer to stabilize the feature distribution and avoid gradient problems.

[0105] The geometric feature mapping network employs two fully connected layers: the first layer is 18 to 1024 dimensions (activation function LeakyReLU, negative slope 0.01), and the second layer is 1024 to 2048 dimensions (activation function LeakyReLU). By progressively increasing the dimensionality, the expression of geometric features in high-dimensional space is better expanded.

[0106] Temperature feature mapping network: adopts a "fully connected + dropout" structure, with the first layer being 32 to 1024 dimensions (activation function ReLU) and the second layer being 1024 to 2048 dimensions (activation function Tanh). The dropout rate is set to 0.2 to prevent overfitting while retaining key information of temperature features.

[0107] S402. Calculate the weights of the mapped texture features, mapped geometric features, and mapped temperature features respectively.

[0108] In this embodiment, a targeted attention weight network is used. The three mapped 2048-dimensional features are input, and three weights are output, which sum to 1.

[0109] S403. Based on the weights of the mapped texture features, mapped geometric features, and mapped temperature features, the mapped texture features, mapped geometric features, and mapped temperature features are fused to obtain fused features.

[0110] In this embodiment, L2 normalization is performed on each feature before weighting to avoid the fusion result being affected by differences in feature numerical ranges. After fusion, batch normalization is performed to make the mean of the fused features 0 and the variance 1, which facilitates subsequent processing by the recognition model.

[0111] In this embodiment, by mapping multimodal features to a 2048-dimensional embedding space, the expressive power and fusion effect of the features are further enhanced, demonstrating significant advantages. The 2048-dimensional high-dimensional space can more fully retain the detailed information of the original features, and the dynamic weight calculation mechanism combined with high-dimensional features can more accurately capture the differences in the importance of each feature under different scenarios, making the weight allocation more in line with actual recognition needs. In addition, the weight-based fusion process takes advantage of the sufficient capacity of the 2048-dimensional space to give full play to the complementarity of the three features. The fused features can more comprehensively reflect the texture, geometry and temperature characteristics of the package, effectively improving the recognition ability of complex adhesion scenarios.

[0112] Please see Figure 5 A fifth embodiment of a method for identifying multiple packages adhering to each other on a logistics conveyor belt, as described in this invention, includes:

[0113] S501, Reshape the fused features into a feature map format that is compatible with the adhesive package recognition model.

[0114] In this embodiment, the sticky package recognition model typically requires a feature map of a specific size as input, such as a feature map (height, width, number of channels) with a size of (512, 512, 3). This size is the input specification that the model adapts to during training, allowing each layer of the model's network to extract features better.

[0115] Fusion feature morphology analysis: The fused feature obtained after fusion is a 2048-dimensional vector. This one-dimensional vector form cannot be directly processed by the model that requires feature map input, and morphology transformation must be performed.

[0116] First, the reshaping parameters are calculated based on the feature map size required by the adhesive package recognition model. Assuming the model requires 3 feature map channels, the 2048-dimensional fused features are distributed among the 3 channels according to certain rules. An approximately equal distribution method is adopted: the first channel is allocated 683 dimensions, the second channel 683 dimensions, and the third channel 682 dimensions. This distribution method can ensure that the feature dimensions of each channel are similar to the greatest extent, avoiding the impact of insufficient features in one channel on information expression. Next, the feature vector of each channel is reshaped. Taking the first channel as an example, the 683-dimensional feature vector is reshaped into a (512, 512) two-dimensional matrix using a bilinear interpolation algorithm. Bilinear interpolation calculates the value of the unknown point by weighted averaging of the surrounding four known points, which can better preserve the distribution trend and key information of the features during the reshaping process and reduce feature distortion caused by dimension transformation. Similarly, the same operation is performed on the 683-dimensional feature vector of the second channel and the 682-dimensional feature vector of the third channel, respectively, to obtain the corresponding (512, 512) two-dimensional matrices.

[0117] Finally, the two-dimensional matrices of the three channels are combined to form a feature map format of (512, 512, 3). This feature map perfectly matches the input specifications of the adhesive package recognition model, preserving the rich information in the 2048-dimensional fused features while being effectively parsed and processed by the model, laying a solid foundation for subsequent adhesive package recognition.

[0118] S502. Input the reshaped fusion features into the adhesive package recognition model and obtain the detection results output by the adhesive package recognition model. The detection results include bounding box coordinates, segmentation mask and confidence score.

[0119] In this embodiment, the sticky package recognition model processes features sequentially through a feature extraction layer, a spatial attention module, a pyramid pooling module, and an output layer. The feature extraction layer further extracts deep semantic information from the fused features; the spatial attention module can specifically focus on key spatial features of the sticky package area; the pyramid pooling module fuses features of different scales to adapt to packages of different sizes; and the output layer generates the final detection result through a series of convolution and activation operations.

[0120] The sticky package recognition model outputs the bounding box coordinates of each sticky package through regression calculation, usually represented in the form of (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left corner of the bounding box and (x2, y2) is the coordinate of the lower right corner of the bounding box. These coordinates are based on the pixel coordinates of the input feature map and can be converted into physical coordinates according to the actual scenario.

[0121] The stuck package recognition model uses semantic segmentation to assign a category label (belonging to a package or background) to each pixel in the feature map, thus forming a segmentation mask. The segmentation mask is a two-dimensional matrix with the same size as the input feature map. Each element in the matrix represents the category of the corresponding pixel. The segmentation mask clearly shows the pixel-level outline of each package, especially the boundaries of the stuck regions.

[0122] The adhesive package recognition model calculates a confidence score for each detection result based on the discriminative power of the features and the uncertainties in the processing. The score ranges from 0 to 1. A higher confidence score indicates a higher reliability of the detection result. For example, a confidence score of 0.9 indicates that the model is 90% confident that the detection result is correct.

[0123] In this embodiment, by reshaping the fused features into a feature map format adapted to the model, the problem of mismatch between feature shape and model input requirements is solved, ensuring that the fused multimodal features can be effectively parsed by the model. This avoids feature information loss or model parsing errors caused by format incompatibility, laying the foundation for the model to achieve optimal performance. In addition, the three types of detection results output by the adhesive package recognition model—boundary box coordinates, segmentation mask, and confidence score—provide key information about adhesive packages from different dimensions. The bounding box enables rapid localization, the segmentation mask enables pixel-level fine segmentation, and the confidence score provides a quantitative assessment of the reliability of the results. The combination of the three makes the detection results both comprehensive and accurate.

[0124] Please see Figure 6 The sixth embodiment of a method for identifying multiple packages adhering to each other on a logistics conveyor belt, as described in this invention, includes:

[0125] S601. If the confidence level exceeds the preset confidence threshold, extract the specific location and segmentation information of the stuck packages from the bounding box coordinates and the segmentation mask.

[0126] In this embodiment, considering the tolerance for false alarm rates in the logistics scenario, a confidence threshold of 0.7 is preset (after extensive experimental verification, the false alarm rate is <5% and the false negative rate is <3% at this threshold). When the confidence level of the detection result is >0.7, it indicates the presence of stuck packages, and the process proceeds to the information extraction stage.

[0127] In this embodiment, the bounding box coordinates (x1, y1, x2, y2, pixel coordinates) are converted to the physical coordinates of the conveyor belt. Specifically, the mapping relationship between pixel coordinates and physical coordinates is established through camera calibration parameters (intrinsic parameter matrix, extrinsic parameter matrix), and the center position of the adhered package is calculated: the center of the physical coordinates of the bounding box ((X1+X2) / 2, (Y1+Y2) / 2) is taken as the main aiming point of the operator or the robotic arm.

[0128] Physical coordinates (X,Y) = extrinsic parameter matrix × inverse intrinsic parameter matrix × pixel coordinates (x,y,1)

[0129] In the formula, X is the position in the width direction of the conveyor belt (unit: meters), and Y is the position in the running direction of the conveyor belt (unit: meters), ensuring that the positioning error is <5cm.

[0130] In this embodiment, the segmentation information extraction is as follows:

[0131] First, the degree of adhesion is determined by calculating the percentage of adhered pixels (number of pixels in the adhered area / total number of pixels in both packages). If the percentage is >30%, it is considered severe adhesion; 10%-30% is considered moderate adhesion; and <10% is considered mild adhesion.

[0132] Next, the contact edge pixels of the two packages in the segmentation mask are converted into physical lengths, accurate to the centimeter level, to provide a difficulty reference for the separation operation. For example, adhesions with edge lengths > 50cm are more difficult to separate.

[0133] S602. Determine the warning level based on the confidence level.

[0134] In this embodiment, the first-level warning (emergency) is: confidence level > 0.9, which corresponds to high reliability of severe adhesion (adhesion pixel ratio > 30%), requiring immediate intervention (such as pausing the local conveyor belt, starting the robotic arm separation) to avoid packages blocking the sorting port;

[0135] Level 2 warning (relatively urgent): 0.8 < confidence level ≤ 0.9, corresponding to moderate adhesion (10% < adhesion pixel percentage ≤ 30%), which needs to be handled within 30 seconds (e.g., notify nearby operators for priority handling);

[0136] Level 3 warning (routine): 0.7 < confidence level ≤ 0.8, corresponding to mild adhesion (adhesive pixel ratio ≤ 10%), which can be included in routine inspection and processing (e.g., summarizing every 5 minutes and handling centrally).

[0137] S603. Generate warning information based on the warning level, the specific location of the stuck packages, and the segmentation information.

[0138] In this embodiment, the warning information includes the following:

[0139] Basic information: warning level (e.g., "Level 1 Warning"), generation time (accurate to the second), associated package ID (matched through the previous package traceability system);

[0140] Location information: physical coordinates of the attached package (e.g., "conveyor belt width 0.8 meters, running direction 12.5 meters"), distance relative to the sorting node (e.g., "3 meters from sorting port 1, expected to arrive in 10 seconds");

[0141] Adhesion details: Adhesion type (e.g., "partial overlap"), adhesion area (e.g., "800cm²") 2 (e.g., "65cm"), contact edge length;

[0142] Handling recommendations: Provide operational guidance based on the warning level and adhesion details. For example, for a Level 1 warning, it is recommended to "immediately start the No. 2 robotic arm for separation" and for a Level 3 warning, it is recommended to "manually separate the parts after the inspection personnel arrive".

[0143] In this embodiment, a confidence threshold is used as the filtering condition to ensure that warnings are generated only based on highly reliable detection results. This effectively filters out false positives with low confidence (such as false adhesion identification caused by feature ambiguity), increasing the accuracy of warnings to over 95%. This avoids invalid warnings interfering with the sorting process. Furthermore, by classifying warning levels by confidence, the system can take differentiated responses based on the urgency of the adhesion situation, improving processing efficiency by 40% compared to single-level warnings. In addition, the warning information integrates the specific location (physical coordinates) and segmentation information (adhesion area, edge details) of the adhered packages, providing operators or automated equipment with precise handling basis. This avoids the ambiguity of traditional warnings that "only alarm without specifying the location," significantly shortening problem handling time.

[0144] The above describes the method for identifying multiple packages adhering to a logistics conveyor belt in an embodiment of the present invention. The following describes the apparatus in an embodiment of the present invention. Please refer to [link / reference]. Figure 7 The embodiments of the multi-package adhesion identification device for logistics conveyor belts in this invention include:

[0145] Training module 701 is used to construct an improved PSPNet model based on the spatial attention mechanism and train the improved PSPNet model to obtain an adhesive package recognition model.

[0146] The extraction module 702 is used to acquire RGB images, depth images and infrared thermal images of packages on the logistics conveyor belt, and extract the texture features of the RGB images, the geometric features of the depth images and the temperature features of the infrared thermal images, respectively.

[0147] The fusion module 703 is used to fuse the texture features, the geometric features, and the temperature features to obtain fused features;

[0148] The recognition module 704 is used to input the fused features into the adhesive package recognition model and obtain the detection results output by the adhesive package recognition model. The detection results include bounding box coordinates, segmentation mask and confidence score.

[0149] The generation module 705 is used to extract the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold, and generate warning information based on the specific location and segmentation information of the stuck package.

[0150] In this embodiment, the training module 701 includes: a construction unit 7011, used to construct an improved PSPNet model based on a spatial attention mechanism; an acquisition unit 7012, used to acquire historical multimodal data of a logistics conveyor belt, and construct a training set and a test set based on the historical multimodal data; and a training unit 7013, used to train the improved PSPNet model using ten-fold cross-validation based on the training set to obtain a trained PSPNet model, and to test the trained PSPNet model using the test set. When the test result is qualified, the trained PSPNet model is used as a model for recognizing sticky packages.

[0151] In this embodiment, the extraction module 702 includes: a first extraction unit 7021, used to acquire an RGB image of a package, a depth image of a package, and an infrared thermal image of a package from a logistics conveyor belt; to extract a region of interest from the RGB image of the package; and to extract texture features of the RGB image of the package from the extracted region of interest; a second extraction unit 7022, used to convert the depth image of the package into point cloud data; and to extract geometric features of the depth image of the package based on the point cloud data; and a third extraction unit 7023, used to convert the grayscale value of the infrared thermal image of the package into a temperature value; and to extract the temperature features of the infrared thermal image of the package based on the temperature value.

[0152] In this embodiment, the fusion module 703 includes: a mapping unit 7031, used to map the texture features, the geometric features, and the temperature features to a 2048 embedding space to obtain mapped texture features, mapped geometric features, and mapped temperature features; a calculation unit 7032, used to calculate the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features respectively; and a fusion unit 7033, used to fuse the mapped texture features, the mapped geometric features, and the mapped temperature features based on the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features to obtain fused features.

[0153] In this embodiment, the recognition module 704 includes: a reshaping unit 7041, used to reshape the fused features into a feature map format adapted to the adhesive package recognition model; and a recognition unit 7042, used to input the reshaped fused features into the adhesive package recognition model and obtain the detection results output by the adhesive package recognition model, wherein the detection results include bounding box coordinates, segmentation mask and confidence score.

[0154] In this embodiment, the generation module 705 includes: a fourth extraction unit 7051, used to extract the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold; a judgment unit 7052, used to determine the warning level based on the confidence level; and a generation unit 7053, used to generate warning information based on the warning level, the specific location and segmentation information of the stuck package.

[0155] In this embodiment, the PSPNet model, by introducing a spatial attention mechanism, can specifically focus on the key spatial features of the package adhesion area, strengthen the feature extraction of sensitive areas such as package edges and contact boundaries, effectively suppress the interference of irrelevant information such as conveyor belt background and noise, and improve the model's feature discrimination ability in complex adhesion scenarios. Moreover, the acquisition and feature extraction of multimodal images (RGB, depth, infrared thermal images) characterize package attributes from multiple dimensions such as texture, geometry, and temperature, avoiding the limitations of a single data source. The feature fusion strategy further integrates the advantages of each modality, further improving the model's adaptability to complex adhesion scenarios. In addition, by combining confidence judgment and accurate location information extraction, timely and effective early warnings can be generated, providing reliable technical support for the automation of logistics sorting, significantly reducing the cost of manual intervention, and improving sorting efficiency and accuracy.

[0156] Figure 7 The structure of the multi-package adhesion identification device for logistics conveyor belts shown does not constitute a limitation on the multi-package adhesion identification device for logistics conveyor belts, and can implement the steps of the multi-package adhesion identification method for logistics conveyor belts provided in the above-described method embodiments.

[0157] above Figure 7 The multi-package adhesion identification device for logistics conveyor belts in this embodiment of the invention is described in detail from the perspective of modular functional entities. The multi-package adhesion identification device for logistics conveyor belts in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0158] Figure 8This is a schematic diagram of the structure of a multi-package adhesion identification device for a logistics conveyor belt provided in an embodiment of the present invention. The device 800 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 810 (e.g., one or more processors) and a memory 820, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 833 or data 832. The memory 820 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown), each module including a series of instruction operations on the device 800. Furthermore, the processor 810 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media on the device 800.

[0159] Device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input / output interfaces 860, and / or one or more operating systems 831, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0160] This invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the method for identifying multiple packages adhering to a logistics conveyor belt.

[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0162] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying multiple packages stuck together on a logistics conveyor belt, characterized in that, The method for identifying multiple packages stuck together on a logistics conveyor belt includes: An improved PSPNet model was constructed based on the spatial attention mechanism, and the improved PSPNet model was trained to obtain an adhesive package recognition model. Acquire RGB images, depth images, and infrared thermal images of packages from a logistics conveyor belt, and extract texture features from the RGB images, geometric features from the depth images, and temperature features from the infrared thermal images, respectively. The texture features, geometric features, and temperature features are fused to obtain a fused feature; The fused features are input into the adhesive package recognition model, and the detection results output by the adhesive package recognition model are obtained. The detection results include bounding box coordinates, segmentation mask, and confidence score. If the confidence level exceeds a preset confidence threshold, the specific location and segmentation information of the stuck packages are extracted from the bounding box coordinates and the segmentation mask, and an early warning message is generated based on the specific location and segmentation information of the stuck packages.

2. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 1, characterized in that, An improved PSPNet model based on the spatial attention mechanism is constructed and trained to obtain an adhesive package recognition model. An improved PSPNet model is constructed based on the spatial attention mechanism; Acquire historical multimodal data of the logistics conveyor belt, and construct training and testing sets based on the historical multimodal data; Based on the training set, the improved PSPNet model is trained using the ten-fold cross-validation method to obtain the trained PSPNet model. The trained PSPNet model is then tested using the test set. When the test result is qualified, the trained PSPNet model is used as the adhesive package recognition model.

3. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 2, characterized in that, The improved PSPNet model based on the spatial attention mechanism includes: The PSPNet model is constructed by including a feature extraction layer, a pyramid pooling module, and an output layer. The pyramid pooling module adopts a progressively dilated convolutional pyramid. An improved PSPNet model is obtained by constructing a spatial attention module and embedding it between the feature extraction layer and the pyramid pooling module. The feature extraction layer is used to form a fused feature map based on the fused features. The spatial attention module is used to receive the fused feature map, perform weighted processing on the fused feature map, and output a weighted feature map. The progressively dilated convolutional pyramid is used to perform multi-scale fusion on the weighted feature map and output the multi-scale fused feature map to the output layer. After processing the multi-scale fused feature map, the output layer outputs the bounding box coordinates, segmentation mask, and confidence score.

4. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 1, characterized in that, The process of acquiring RGB images, depth images, and infrared thermal images of packages from the logistics conveyor belt, and extracting texture features from the RGB images, geometric features from the depth images, and temperature features from the infrared thermal images, includes: Acquire RGB images, depth images, and infrared thermal images of packages from a logistics conveyor belt; extract regions of interest from the RGB images of the packages; and extract texture features from the extracted regions of interest. The package depth image is converted into point cloud data, and the geometric features of the package depth image are extracted based on the point cloud data; The grayscale values ​​of the packaged infrared thermal image are converted into temperature values, and the temperature features of the packaged infrared thermal image are extracted based on the temperature values.

5. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 1, characterized in that, The process of fusing the texture features, geometric features, and temperature features to obtain a fused feature includes: The texture features, geometric features, and temperature features are mapped to a 2048 embedding space to obtain mapped texture features, mapped geometric features, and mapped temperature features. Calculate the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features respectively; Based on the weights of the mapped texture features, the mapped geometric features, and the mapped temperature features, the mapped texture features, the mapped geometric features, and the mapped temperature features are fused to obtain fused features.

6. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 1, characterized in that, The fused features are input into the adhesive package recognition model, and the detection results output by the adhesive package recognition model are obtained. The detection results include bounding box coordinates, segmentation mask, and confidence score, including: The fused features are reshaped into a feature map format that is compatible with the adhesive package recognition model; The reconstructed fusion features are input into the adhesive package recognition model, and the detection results output by the adhesive package recognition model are obtained. The detection results include bounding box coordinates, segmentation mask, and confidence score.

7. The method for identifying multiple packages adhering to each other on a logistics conveyor belt according to claim 1, characterized in that, If the confidence level exceeds a preset confidence threshold, the specific location and segmentation information of the stuck packages are extracted from the bounding box coordinates and the segmentation mask, and warning information is generated based on the specific location and segmentation information of the stuck packages, including: If the confidence level exceeds a preset confidence threshold, the specific location and segmentation information of the adhered packages are extracted from the bounding box coordinates and the segmentation mask. The warning level is determined based on the confidence level. Warning information is generated based on the warning level, the specific location of the stuck package, and the segmentation information.

8. A multi-package adhesion identification device for a logistics conveyor belt, characterized in that, include: The training module is used to build an improved PSPNet model based on the spatial attention mechanism and train the improved PSPNet model to obtain the sticky package recognition model. The extraction module is used to acquire RGB images, depth images, and infrared thermal images of packages on the logistics conveyor belt, and to extract the texture features of the RGB images, the geometric features of the depth images, and the temperature features of the infrared thermal images, respectively. The fusion module is used to fuse the texture features, the geometric features, and the temperature features to obtain fused features; The recognition module is used to input the fused features into the adhesive package recognition model and obtain the detection results output by the adhesive package recognition model. The detection results include bounding box coordinates, segmentation mask and confidence score. The generation module is used to extract the specific location and segmentation information of the stuck package from the bounding box coordinates and the segmentation mask if the confidence level exceeds a preset confidence threshold, and generate warning information based on the specific location and segmentation information of the stuck package.

9. A multi-package adhesion identification device for a logistics conveyor belt, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform the steps of the multi-package adhesion identification method for a logistics conveyor as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the various steps of the multi-package adhesion identification method for logistics conveyor belts as described in any one of claims 1-7.

Images