Parcel and face sheet detection method based on multi-scale feature bidirectional fusion

Abstract

The application discloses a logistics parcel and face sheet detection method based on bidirectional fusion of multi-scale features, and belongs to the technical field of computer vision; the application introduces an adaptive weighted feature pyramid module into the neck network to replace the traditional fixed splicing mode, realizes adaptive dynamic fusion of multi-scale features, and simultaneously introduces a gating context interaction module to dynamically balance the global context information and local details of high-level features through learnable parameters; in addition, the P2 level shallow features with high resolution in the backbone network are introduced into the neck network for enhanced fusion, so that the defect of small target detail loss is compensated; the application effectively solves the problems of difficult parcel stacking occlusion identification, high small-size parcel and face sheet missing detection rate in the prior art, and significantly improves the detection accuracy and robustness of parcels and face sheets in a logistics complex scene under the premise of ensuring real-time inference speed.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features. Background Technology

[0002] With the rapid development of e-commerce and the explosive growth of logistics volume, the demand for automation and intelligent management of logistics parcels is becoming increasingly urgent. Computer vision-based target detection technology, with its high efficiency and non-contact characteristics, has been widely applied to parcel identification, precise positioning, and automated sorting tasks in logistics scenarios. Although computer vision-based target detection technology has achieved initial applications in the field of logistics parcel identification in recent years, research on the automatic detection and sorting of parcels and waybills in complex logistics scenarios is still in its early stages.

[0003] Existing solutions largely rely on rule-based template matching or fixed scene presets. For example, the Chinese patent document "A Logistics Parcel Detection Method Based on a Lightweight Target Detection Model" (CN115375937B, August 5, 2025) uses collected conveyor belt images to create a target detection dataset, which is insufficient to address challenges in logistics scenarios such as parcel stacking and occlusion, tilted and damaged waybills, drastic changes in lighting, and real-time requirements under high-speed movement. Furthermore, most detection methods suffer from poor robustness and limited positioning accuracy when facing these challenges; and the detection success rate significantly decreases when dealing with parcels that are tilted or have damaged waybills, failing to achieve accurate classification and efficient sorting of items.

[0004] Therefore, in actual logistics scenarios, package and waybill inspection still faces the following technical challenges:

[0005] 1) Significant scale variations exist in logistics parcels. Parcels vary widely in size, ranging from document bags and small envelopes less than 1 cm thick to large express delivery boxes with sides exceeding half a meter. Existing detection algorithms often struggle to simultaneously maintain the semantic integrity of large-scale targets and preserve the details of small-scale targets during feature extraction and fusion. In particular, the neck network often employs simple feature concatenation, failing to fully utilize the complementary information between different feature levels. This leads to missed and false detections of small-scale parcels, impacting overall detection accuracy.

[0006] 2) Severe stacking and occlusion issues exist among packages. On high-speed conveyor belts or in densely packed sorting centers, packages often stick together and stack, creating complex spatial occlusion relationships that blur target boundaries and result in missing local features. Under these conditions, detection models struggle to accurately distinguish the boundaries of individual packages, easily misclassifying multiple stacked packages as a single target or completely missing occluded targets. While existing RT-DETR (Real-Time Detection Transformer) incorporates an AIFI module for global context modeling, its output lacks an effective dynamic balancing mechanism with the original features, potentially leading to excessive smoothing of local details based on global information.

[0007] 3) Waybill detection presents unique challenges. As a crucial information carrier of packages, waybills are typically small in size, their placement is not fixed, and they are easily affected by wrinkles, reflections, and partial occlusion. Existing detection algorithms often struggle to accurately detect both packages and waybills simultaneously. Training two separate models increases computational overhead and deployment complexity; if the same feature extraction network is used, the small-scale features of the waybill are easily lost in the deep network, leading to a high false negative rate for waybills, which in turn affects subsequent OCR information recognition.

[0008] 4) Logistics scenarios place extremely high demands on real-time detection. Automated sorting lines require detection algorithms with millisecond-level inference speeds to match the high-speed movement of conveyor belts and the rapid grasping of robotic arms. While some existing high-precision detection algorithms have high accuracy, their high computational complexity makes them difficult to deploy in real-time on edge devices, failing to meet the demands of actual production cycles. Although RT-DETR achieves a good balance between speed and accuracy, its neck network still suffers from issues such as a single fusion method and insufficient utilization of shallow features, limiting its further application in logistics scenarios.

[0009] RT-DETR, as a real-time end-to-end target detector, achieves a good balance between speed and accuracy by decoupling intra-scale interactions and cross-scale fusion. However, existing RT-DETR neck networks often employ simple feature concatenation methods, failing to introduce dynamic weight modulation mechanisms to fully utilize complementary information between features at different levels. Furthermore, they underutilize shallow high-resolution features in the backbone network, directly discarding P2-level features without participating in the neck fusion process, leading to the gradual dilution of detailed information about small target packages during feature transfer. Additionally, while its AIFI module can capture global context, it lacks a dynamic balancing mechanism with the original features, easily causing excessive smoothing of local details. These limitations limit the accuracy of existing RT-DETR in logistics package detection, making it difficult to meet practical application requirements. Summary of the Invention

[0010] The technical solution of this invention is used to solve the problem of how to improve the accuracy and real-time performance of logistics parcel detection.

[0011] The present invention solves the above-mentioned technical problems through the following technical solutions: This invention provides a method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features, comprising the following steps: Acquire images from logistics sorting scenarios and construct a dataset of logistics parcel and waybill images; An improved RT-DETR network model is constructed, comprising a backbone network, an improved neck network, and a detection head network; wherein, the construction of the improved neck network includes: The different levels of features output from the backbone network are projected onto a unified number of channels to obtain the corresponding projected features. Global context modeling is performed on the highest-level projection features, and the modeled global context features are dynamically fused with the highest-level projection features through a learnable gating mechanism to generate gated fused features. A top-down multi-scale fusion is performed on the gated fusion features. A dynamic weight modulation mechanism is introduced during the fusion process. After upsampling the high-level features, they are adaptively weighted and fused with the low-level features. P2-level shallow features from the backbone network are introduced, and after downsampling, they are adaptively weighted and fused with the same-level features from the top-down fusion process to obtain shallow enhanced features. Perform bottom-up multi-scale fusion on shallow enhancement features, downsample low-level features and adaptively weighted fuse them with high-level features to generate the final multi-scale fused feature map. The images of the logistics packages and waybills to be detected are input into the trained improved RT-DETR network model. The detection head network makes predictions based on the final multi-scale fusion feature map and outputs the detection results of the packages and waybills.

[0012] Furthermore, the dynamic fusion of the global context features obtained from modeling and the highest-level projected features through a learnable gating mechanism specifically satisfies the following formula:

[0013] in, The characteristics after fusion The global context features output by AIFI α represents the original projected features, and α is a learnable gating parameter.

[0014] Furthermore, the adaptive weighted fusion is implemented through an adaptive weighted feature pyramid module, which dynamically generates fusion weights based on the content of the input features. The calculation process satisfies the following formula:

[0015] in, For dynamic weights, For integrated output, For fully connected layer operations, This is a global average pooling operation. It is the Sigmoid activation function. For input features, This indicates multiplication by channel. It is a constant.

[0016] Furthermore, the P2-level shallow features introduced into the backbone network, after downsampling, are adaptively weighted and fused with the same-level features from the top-down fusion process, specifically including: Extract P2-level high-resolution feature maps from the backbone network output; The number of channels in the P2-level high-resolution feature map is compressed to a preset uniform dimension through convolution operations; Perform a downsampling operation with a step size of 2 on the channel-compressed P2 level feature map to align its spatial resolution with that of the P3 level; The aligned features and the P3-level intermediate features generated during the top-down fusion process are input into the adaptive weighted feature pyramid module, and fusion is achieved through dynamic weight modulation.

[0017] Furthermore, after each execution of the adaptive weighted fusion, the improved neck network also includes a step of using the RepC3 module to perform feature processing on the fused features; the RepC3 module adopts a structure containing multi-branch convolutions during the training phase to enhance feature representation capabilities, and fuses the multi-branch convolutions into a single-path convolution through structural reparameterization technology during the inference phase to reduce inference computational overhead.

[0018] Furthermore, the feature processing of the RepC3 module during the training phase satisfies the following formula:

[0019] Where X is the input feature. This indicates a block containing multiple reparameterized convolutional blocks, each consisting of... , Feature extraction branches consisting of identity branches; This indicates splicing along the channel dimension; finally, through... Perform feature fusion.

[0020] Furthermore, the detection head network employs a decoder based on the Transformer architecture and iteratively optimizes the target query through a deformable attention mechanism, which satisfies the following formula:

[0021] in, Target query features, For reference point location, For multi-scale feature maps, The number of sampling points. For the number of attention heads, For attention weights, For learnable sampling offset, For deformable attention functions, Let be the output projection weight matrix for the k-th sampling point. The projection weight matrix is ​​used to represent the value of the k-th sampling point.

[0022] Furthermore, the training process of the improved RT-DETR network model employs loss functions including the Focal Loss loss function for classification tasks and the GIoU loss function for regression tasks.

[0023] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the above-described method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features, and the processor is configured to execute the program stored in the memory.

[0024] The present invention also provides a storage medium storing a computer program, which, when run by a processor, executes the steps of the above-described method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features.

[0025] The beneficial effects of this invention are as follows: This invention collects package images in logistics sorting scenarios to construct a logistics package image dataset; it builds an improved RT-DETR model, which includes a backbone network, an improved neck network, and a detection head network. The improved neck network employs an Adaptive Weighted Feature Pyramid (AWFPN) structure to achieve adaptive weighted fusion of multi-scale features and introduces a Gated Context Interaction Module (GCIM) to achieve autonomous balance between global context and local details. Simultaneously, a shallow feature enhancement branch fuses high-resolution P2 features from the backbone network with a top-down path to improve small target detection capabilities. The improved RT-DETR model is trained using the dataset to obtain a trained improved RT-DETR model. The T-DETR model inputs real-time acquired images of logistics packages into a pre-trained improved RT-DETR model, thereby outputting the detection results of packages and waybills. This invention effectively addresses the problems of large differences in package scale, severe stacking occlusion, and difficulty in detecting small targets in logistics scenarios. Through the synergistic effect of adaptive weighted feature pyramid and gating context interaction module, it achieves high-precision package and waybill detection and category matching while maintaining real-time detection performance, providing accurate visual perception results for automated logistics sorting systems. By introducing a dynamic weight modulation mechanism to achieve adaptive weighted fusion of multi-scale features, and combining shallow feature enhancement with gating context interaction, the accuracy and real-time performance of logistics package detection are improved. Attached Figure Description

[0026] Figure 1 This is an overall flowchart of the logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion of the present invention; Figure 2 This is a schematic diagram of the improved RT-DETR network model structure of the present invention; Figure 3 This is a schematic diagram of the GCIM module structure of the present invention; Figure 4 This is a schematic diagram of the AWFPN bidirectional feature fusion module structure of the present invention; Figure 5 This is a schematic diagram of the RepC3 module structure of the present invention; Figure 6 This is a comparison chart showing the inspection effects of packages and waybills before and after the improvements of this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 As shown, this embodiment provides a method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features, specifically including the following steps: Step 1: Obtain package images in a logistics sorting scenario and construct a logistics package image dataset. Images of packages were acquired in a logistics sorting center conveyor belt environment using industrial cameras and supplemental lighting to ensure image clarity. Annotation tools were used to add bounding boxes and category labels to the packages and shipping labels in the images. The annotated dataset was then divided into training, validation, and test sets in an 8:1:1 ratio. To enhance the model's generalization ability, data augmentation was performed on the training set images, including random flipping, random rotation, random scaling, color jittering, and mosaic enhancement.

[0029] Step 2: Construct an improved RT-DETR network model like Figure 2 As shown, the improved RT-DETR network model constructed in this embodiment includes a backbone network, an improved neck network, and a detection head network.

[0030] 2.1 Backbone Network The backbone network employs a hierarchical downsampling structure, sequentially outputting feature maps at four scales: P2, P3, P4, and P5. Largest scale, weakest semantics It has the smallest scale and the strongest semantics. Specifically, the backbone network contains the following layers: Layers 0-3: Initial downsampling layers, which gradually reduce the size of the input image; Layer 4: Output The scale feature map has a resolution of up to 1 / 4 of the input image size, preserving rich location and detail information; Layer 5: Output Scale feature map, with a resolution of 1 / 8 of the input image; Layer 6: Output Scale feature map, with a resolution of 1 / 16 of the input image, enhances semantic information; Layer 7: Output Scale feature maps have the lowest resolution but the strongest semantic information.

[0031] The backbone network uses BasicBlock as the basic feature extraction module. Multiple BasicBlocks are stacked to gradually increase the number of channels and decrease the spatial resolution. Let the input image size be... The spatial dimensions of the feature maps at each scale are as follows: , , , This multi-scale structure can take into account both the semantic information of large targets and the detailed information of small targets.

[0032] 2.2 Improved Neck Network The neck network is used to fuse and enhance the multi-scale feature maps output by the backbone network. The improved neck network in this embodiment adopts a multi-scale bidirectional feature fusion structure, which is composed of a feature projection unit, a global context modeling unit, a top-down fusion unit, a shallow feature enhancement unit, a bottom-up fusion unit, and a feature processing unit connected in series.

[0033] The feature projection unit is used to project features from different levels output by the backbone network to a uniform number of channels. This operation can align feature dimensions at different levels, providing a uniform-scale feature representation for subsequent fusion.

[0034] In this embodiment, the feature projection unit consists of three parallel Conv[1x1] units, which respectively project the output of the backbone network. , , The feature maps at three scales are subjected to channel transformation, compressing their respective channel counts to a uniform 256. Let the output of the backbone network be the... Layer feature map is The number of channels is transformed to a uniform dimension using Conv[1x1]. The resulting projected feature map is as follows:

[0035] in, This represents the feature map of the i-th layer output by the backbone network (i takes values ​​of 3, 4, and 5 respectively, corresponding to...). , , (hierarchy) , , These represent the number of channels, height, and width of the feature map, respectively. This represents a 1×1 convolution operation; This is the unified number of channels after projection (256 in this embodiment); The feature map obtained after projection has a spatial resolution. It remains unchanged.

[0036] like Figure 3 As shown, the global context modeling unit employs a Gated Context Interaction Module (GCIM), positioned after the highest-level feature (P5). It is used for global context modeling of high-level semantic features and dynamically fuses with the original projected features through a learnable gating mechanism, enhancing the model's ability to understand the spatial relationships of stacked wrappers. The GCIM module is based on a Transformer encoder structure using Attention-based Intra-scale Feature Interaction (AIFI), and its computation process is as follows: First, project the highest level features. Flattened into a sequence ,in And incorporate learnable positional encoding :

[0037] Then, the sequences are interacted with through a multi-head self-attention mechanism. Let... Depend on Obtained through linear transformation, For each dimension of the attention head, the attention is calculated as follows:

[0038] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the scaling factor (usually equal to the number of channels divided by the number of heads), softmax() is the normalization exponential function, and Attention() is the single-head attention function in the multi-head self-attention mechanism, used to calculate the weighted aggregation of each position in the sequence with respect to other positions. The output of the multi-head attention is then fed into a feedforward network (FFN) after residual connections and layer normalization (Add & Norm), finally outputting the enhanced features. Then... Reconstructing back to the spatial dimension yields the AIFI output features. Furthermore, the GCIM module introduces a learnable gating mechanism to dynamically fuse the AIFI output with the original projection features:

[0039] Where α is a learnable gating parameter, initialized to 0.5, which is adaptively optimized by the network during training; The characteristics after fusion The global context features output by AIFI Original projected features (preserving local details). Fuded features. As the starting point of the top-down path, it enables the network to autonomously balance global contextual information and local detailed features, avoids the AIFI module from over-smoothing key details, and significantly enhances the ability to distinguish stacked wrapper boundaries.

[0040] Through the GCIM module described above, high-level features can effectively capture global dependencies, while retaining local details of the original projected features through a learnable gating mechanism. This achieves an autonomous balance between global context and local features, providing optimized input features that combine semantic and detailed information for subsequent multi-scale feature fusion.

[0041] The top-down fusion unit is used to transfer high-level semantic features to low-level features. For example... Figure 4 As shown, unlike the basic RT-DETR model which uses simple concat concatenation, this unit introduces a dynamic weight modulation mechanism, which consists of multiple adaptive weighted feature pyramid (AWFPN) modules alternating with upsampling operations. Each AWFPN module performs adaptive weighted aggregation on different input features. Figure 4 In GFS, the first letters correspond to Global Average Pooling (GAP), Fully Connected Layer (FC), and the Activation Function Sigmoid.

[0042] Specifically, let the characteristics of high-rise buildings be... Low-level features are First, bilinear interpolation upsampling is used to... Spatial resolution magnified to Same, that is Then the upsampled features are compared with... Input the AWFPN module for weighted fusion:

[0043] The core of the AWFPN module is the dynamic weight modulation mechanism, where the fusion weights are dynamically generated from the input features through global average pooling (GAP) and fully connected layers (FC). Let the input features be... (i=1, 2 correspond to low-level and high-level features respectively), dynamic weights and fusion output The calculation is as follows:

[0044] in, It is the Sigmoid activation function. This indicates multiplication by channel. This is a very small constant used for numerical stability. This mechanism enables the network to adaptively assign dynamic weights to features at different scales based on the content of the input image, replacing the fixed stitching operation in the base model.

[0045] In this embodiment, the top-down path includes two AWFPN modules: the first one integrates the gated fused top-layer features. After upsampling, compared with projected features AAWFPN weighted fusion is performed, and the fused features are processed by the RepC3 module to obtain the enhanced features. The second one will After upsampling, compared with projected features The intermediate features are obtained by performing AWFPN weighted fusion and then processing them through the RepC3 module. .like Figure 5 The diagram shown is a schematic of the RepC3 module structure in this embodiment.

[0046] The aforementioned fusion operation is also applied to each fusion node in the bottom-up path (using downsampling instead of upsampling), forming a complete bidirectional interaction. Through end-to-end training, the network can adaptively assign dynamic weights to features of different scales based on the input image content, achieving effective fusion of multi-scale information.

[0047] The shallow feature enhancement unit is used to introduce high-resolution shallow features from the backbone network to enhance the ability to perceive details of small-scale packages and shipping labels. Unlike the basic RT-DETR model, which directly discards P2 features, this unit introduces the high-resolution P2 feature map output from the second layer of the backbone network after downsampling into the neck network through a bypass.

[0048] Let the feature map output after downsampling in the second layer of the backbone network be... It has the highest resolution (1 / 4 of the input image) and contains rich spatial details. First, its channel count is transformed to a uniform dimension using Conv[1x1]. ,get Then, downsampling is performed using Conv[3x3, s=2] to align its spatial resolution with that of layer P3 (reducing it to 1 / 8 of the input image), resulting in... Finally, Features of P3 level in the top-down path The inputs are weighted and fused using the AWPPN module, and then processed by the RepC3 module to obtain the enhanced features. as follows:

[0049] By introducing shallow high-resolution features, this unit compensates for the deficiency of the basic model in utilizing the details of small-scale packages and shipping labels, significantly enhances the ability to perceive details of small-scale packages and shipping labels, and effectively improves the detection accuracy of small-sized targets.

[0050] The bottom-up fusion unit is used to pass enhanced low-level detail features to higher levels, forming a bidirectional interactive path with the top-down unit to achieve full fusion of multi-scale features. This unit consists of two AWFPN modules alternating with downsampling operations, progressively passing enhanced low-level features to higher levels.

[0051] Specifically, let the enhanced low-level features be... High-level characteristics are The intermediate features at the same level in the top-down path are First, use Conv[3x3, s=2] to... Downsampling to Same spatial resolution, i.e. Then combine it with and The weighted fusion of inputs into the AWFPN module is as follows:

[0052] In this embodiment, the bottom-up path includes two AWFPN modules: the first one enhances the shallow-layer features. After downsampling, compared with projected features and top-down enhancement AWFPN fusion is performed, followed by processing by the RepC3 module to obtain the final low-to-mid-level features. The second one will After sampling, compared with projected features and the gated fusion features output by the GCIM module AAWFPN weighted fusion is performed, followed by processing by the RepC3 module to obtain the final high-level features. .

[0053] The feature processing unit uses the RepC3 module as the core feature processing node. During the training phase, the RepC3 module employs a multi-branch structure including 3×3 convolutions, 1×1 convolutions, and identity mappings to enhance feature representation capabilities. During the inference phase, it uses structural reparameterization technology to fuse the parameters of the multi-branch modules into a single-path convolution, thereby eliminating the additional computational overhead caused by the training branches and achieving inference speed comparable to ordinary convolutions while maintaining detection accuracy.

[0054] The calculation process of the RepC3 module is as follows (training phase):

[0055] Where X represents the input feature; This indicates a block containing multiple reparameterized convolutional blocks, each consisting of... , Feature extraction branches consisting of identity branches; This indicates splicing along the channel dimension; finally, through... Feature fusion is performed during the training phase. The multi-branch structure within the block enables the network to learn richer feature representations; during the inference phase, the multi-branch parameters are fused into a single convolutional kernel using the following formula:

[0056] in, and They are respectively , And the parameters corresponding to the identity mapping, This means filling the low-dimensional kernel to the same size as the high-dimensional kernel. The fused RepC3 module significantly reduces inference computation while maintaining accuracy.

[0057] Through the bidirectional fusion of top-down, shallow enhancement, and bottom-up approaches described above, the neck network ultimately outputs three enhanced feature maps: , and These are used to detect packages and waybills at different scales, providing rich and accurate multi-scale feature representations for subsequent detection heads.

[0058] 2.3 Detection Head Network The detection head network employs an RTDETRDecoder, which includes an encoder and a decoder. The encoder converts the output of the neck network... , and The feature maps at three scales are each enhanced with intra-scale features through independent multi-layer Transformer encoding layers to extract rich feature representations. The decoder iteratively optimizes the target query using a deformable attention mechanism, ultimately generating bounding box coordinates and class confidence. The decoder receives the multi-scale features output from the encoder as keys and initializes 300 learnable target queries, which are then progressively optimized through six Transformer decoding layers. Each decoder layer uses a deformable attention mechanism instead of traditional self-attention, and the calculation of this deformable attention mechanism is shown below:

[0059] in, Target query features, For reference point location, For multi-scale feature maps, The number of sampling points. For the number of attention heads, For attention weights, For learnable sampling offset, This is a deformable attention function used to adaptively sample and aggregate features from a multi-scale feature map based on query features and reference points. Let be the output projection weight matrix for the k-th sampling point. The projection weight matrix is ​​used to represent the value of the k-th sampling point.

[0060] By employing deformable attention, the decoder can focus on sparse keypoints within the bounding region, improving detection efficiency. Finally, a linear layer predicts the bounding box coordinates and class probability distribution for each query.

[0061] The output of the detection head network is shown in the following formula:

[0062] in, The bounding box coordinates of the i-th predicted target Let i be the probability distribution of the category corresponding to the i-th predicted target. To predict the total number of targets in the output, The index for the prediction target.

[0063] Step 3: Training the improved RT-DETR network model The improved RT-DETR network model constructed in step 2 is trained end-to-end using the training set constructed in step 1.

[0064] The training parameters are set as follows: input image size is 640×640, batch size is 16, the optimizer is AdamW, and the initial learning rate is... The learning rate was adjusted using a cosine annealing strategy with a weight decay coefficient of 0.0005, and the training lasted for 200 epochs.

[0065] The total loss function used in the training process includes classification loss and regression loss, where the classification loss uses... The regression loss uses the GIoU loss function.

[0066] The formula for calculating Focal Loss is as follows:

[0067] in, This represents the model's predicted probability of the true class. To balance the weighting factors of positive and negative samples, For focusing parameters, it is usually taken as follows: This loss function can effectively alleviate the imbalance between positive and negative samples, making the model pay more attention to samples that are difficult to classify.

[0068] The formula for calculating the GIoU loss function is as follows:

[0069] in, for Loss function value, To predict the bounding box, For the true bounding box, For inclusion and The smallest closure region; Loss in The addition of a penalty for overlapping regions to the loss statement provides better guidance for bounding box regression.

[0070] The total loss function is the sum of the classification loss and the regression loss. The formula for calculating the total loss function is as follows:

[0071] By jointly optimizing the classification and regression objectives, the model can achieve higher accuracy and robustness in logistics parcel and waybill detection tasks.

[0072] Step 4: Lightweight model deployment and package and shipping label detection The trained improved RT-DETR model is converted into a TensorRT inference engine, accelerated with FP16 quantization, and deployed to NVIDIA Jetson series edge computing devices. By inputting the images of the logistics packages and waybills to be detected, package and waybill detection can be achieved.

[0073] Experimental verification This invention compares several classic object detection networks, including Faster R-CNN, YOLO series, DETR, RT-DETR-R18, and RT-DETRv2-S. The experimental results are shown in Table 1 below.

[0074] Table 1 Comparison of experimental results for various target detection networks

[0075] The table lists the precision (P), recall (R), average precision (mAP50), number of parameters (Para), and frames per second (FPS) of each model, comprehensively evaluating the overall performance of the model from three dimensions: detection accuracy, computational efficiency, and resource consumption. Figure 6 This is a comparison chart of the package and waybill detection effects before and after the improvement in this embodiment of the invention. Figure 6As shown in Table 1, the improved RT-DETR model of this invention achieves optimal detection accuracy: mAP50 reaches 84.9%, precision is 84.5%, and recall is 82.1%, all higher than other comparative models. In particular, compared to the baseline RT-DETR-R18, mAP50 is improved by 2.5 percentage points. Although the improved RT-DETR model of this invention has a larger number of parameters (25.3M) and a lower inference speed (49.6 FPS) than YOLO's lightweight models, this speed fully meets the real-time requirements in actual logistics sorting scenarios. Industrial cameras typically acquire images at 30-60 FPS, while the improved RT-DETR model's 49.6 FPS is far higher than common acquisition frame rates, thus not causing a processing bottleneck. Furthermore, compared to Faster R-CNN and DETR, which have larger parameter counts and slower speeds, the improved RT-DETR model of this invention has significant advantages in both accuracy and speed. Therefore, the improved RT-DETR model of this invention not only outperforms existing mainstream methods in terms of package and waybill detection performance, but also reliably meets the needs of automated logistics sorting systems for real-time detection and efficient processing.

[0076] To further verify the effectiveness of each improved module of the improved RT-DETR model of this invention, ablation experiments were also conducted, and the results are shown in Table 2: Table 2 Ablation Experiment Results

[0077] Ablation experiments validated the effectiveness of each module in the improved RT-DETR model of this invention. Specifically, the Adaptive Weighted Feature Pyramid (AWFPN) module and the shallow feature enhancement P2 branch significantly contributed to the detection accuracy: introducing the AWFPN module alone improved the mAP50 index by 0.8 percentage points, and introducing the P2 branch alone improved the mAP50 index by 1.0 percentage point. When the two work synergistically, the mAP50 improvement reaches 1.9 percentage points, which is better than the arithmetic sum of the improvements of the two modules alone, fully demonstrating the synergistic gain effect of the above modules in multi-scale feature fusion and small target detail enhancement. Based on the above, the improved RT-DETR model of this invention further introduces the Gated Context Interaction (GCIM) module, which achieves a dynamic balance between global contextual information and local detail features, avoiding over-smoothing of key features. Compared with the baseline model, the complete model improves mAP50 by 2.5 percentage points and mAP50:95 by 2.3 percentage points. The above experimental results fully validate the effectiveness of the improved RT-DETR model of this invention and its synergistic gain.

[0078] The improved RT-DETR model of this invention employs a hierarchical downsampling structure in its backbone network, sequentially outputting feature maps at four scales: P2, P3, P4, and P5. Through multi-scale feature extraction, it can better capture the feature information of packages and waybills of different sizes in complex logistics scenarios. Furthermore, the improved neck network introduces an Adaptive Weighted Feature Pyramid (AWFPN) structure, replacing the traditional simple feature concatenation method. Through a dynamic weight modulation mechanism, it achieves adaptive weighted aggregation of multi-scale features, enabling the network to maintain good detection performance even in complex environments, with diverse packages, and small-sized targets. Simultaneously, the shallow feature enhancement branch introduces high-resolution P2 feature maps from the backbone network into the neck network, enhancing the ability to perceive details of small-scale packages and waybills. The Gated Context Interaction (GCIM) module, combined with a learnable gating mechanism, achieves an autonomous balance between global context and local details, strengthening the understanding of spatial relationships among stacked packages. The RepC3 module, as the core feature processing node, uses a multi-branch structure to enhance feature representation during training and fuses it into a single-path convolution through structural reparameterization during inference, improving accuracy while maintaining high inference speed. Ultimately, the RTDETRDecoder detection head outputs the bounding box position and confidence level of the package, providing accurate detection results for the logistics sorting system.

[0079] Compared with the basic RT-DETR model, the improved RT-DETR model of this invention significantly improves detection accuracy while maintaining real-time detection efficiency. It exhibits excellent robustness, especially in scenarios involving small target packages, waybills, and stacked occlusion, and is worthy of widespread application in automated logistics sorting systems.

[0080] This invention replaces the simple feature concatenation method in the original RT-DETR neck network with an Adaptive Weighted Feature Pyramid (AWFPN) structure. By introducing a dynamic weight modulation mechanism, it achieves adaptive weighted aggregation of multi-scale features, significantly improving the feature representation capability for packages of different scales. This structure, through a bidirectional path from top to bottom and bottom to top, allows for full interaction between high-level semantic information and low-level detailed information, effectively solving the problem of large differences in package scale and the coexistence of large and small targets in logistics scenarios. Simultaneously, a shallow feature enhancement branch is introduced to enhance the high-resolution features in the backbone network. Feature maps are introduced into the neck network, and then downsampled and compared with those in the top-down path. Feature fusion enhances the ability to perceive detailed features of small-scale packages and waybills, significantly improving the problem of missed detection of small-sized packages and waybills. Finally, the RTDETRDecoder detection head is used to simultaneously output the bounding box positions and category confidence of packages and waybills, providing accurate detection results for the logistics sorting system.

[0081] The RepC3 module introduced in the neck network of this invention optimizes feature processing efficiency. During the training phase, a multi-branch structure is used to enhance feature representation capabilities, and during the inference phase, structural reparameterization is used to fuse the features into a single-path convolution, achieving a balance between accuracy and speed. The Gated Context Interaction (GCIM) module uses a multi-head self-attention mechanism to capture global dependencies of high-level semantic features and dynamically fuses them with the original projected features through a learnable gating mechanism, achieving an autonomous balance between global context and local details. This avoids excessive smoothing of details of small targets such as single objects, strengthens the understanding of spatial relationships of stacked and wrapped objects, and improves detection robustness in occluded scenes. The dynamic weight modulation mechanism of AWFPN enables dynamic calibration of multi-scale features, strengthens the response to key channel and spatial features in the detection task, and improves the overall quality and efficiency of feature representation.

[0082] This invention constructs a bidirectional multi-scale feature fusion mechanism in the neck network, aiming to deeply fuse multi-scale feature maps from different layers of the backbone network with different receptive fields. This mechanism guides shallow high-resolution P2 feature maps to a detail enhancement branch, focusing on capturing and strengthening the local texture and edge information of packages and faceplates; simultaneously, it guides deep P5 feature maps, processed by the GCIM module, to a semantic enhancement branch, responsible for representing the overall spatial layout and contextual relationships of packages. Subsequently, an AWFPN structure is used to deeply enhance these two feature flows along top-down and bottom-up paths, significantly improving their respective local detail representation and global structure modeling capabilities. To achieve deep complementarity and collaborative optimization between cross-scale features, this invention introduces a dynamic weight modulation mechanism for dynamic calibration: the top-down path guides high-level semantic features to low-level features, enhancing their semantic discriminative ability; simultaneously, the bottom-up path guides enhanced low-level detail features to high-level features, supplementing them with refined positional information. Finally, the RepC3 module efficiently reorganizes the fused cross-scale information, generating a unified feature representation that combines pixel-level discriminative sensitivity with macroscopic spatial robustness. This series of designs significantly enhances the model's stability in detecting packages and waybills of different sizes in complex logistics scenarios.

[0083] This invention significantly improves the detection accuracy and robustness of diverse packages, small-sized targets, and waybills in complex logistics environments. Based on this high-precision detection result, the system not only identifies the package category and matches it with subsequent sorting strategies, but also accurately extracts the waybill area, providing high-quality ROI input for subsequent OCR information recognition. By simultaneously outputting package location and waybill area, it provides a more comprehensive environmental awareness capability for automated logistics sorting systems.

[0084] Example 2 This embodiment provides an electronic device, including a memory and a processor. The memory is used to store a program that supports the processor in executing the logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion in Embodiment 1. The processor is configured to execute the program stored in the memory.

[0085] Example 3 This embodiment provides a storage medium storing a computer program. When the computer program is run by a processor, it executes the steps of the logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion in Embodiment 1.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting logistics parcels and waybills based on bidirectional fusion of multi-scale features, characterized in that, Includes the following steps: Acquire images from logistics sorting scenarios and construct a dataset of logistics parcel and waybill images; An improved RT-DETR network model is constructed, comprising a backbone network, an improved neck network, and a detection head network; wherein, the construction of the improved neck network includes: The different levels of features output from the backbone network are projected onto a unified number of channels to obtain the corresponding projected features. Global context modeling is performed on the highest-level projection features, and the modeled global context features are dynamically fused with the highest-level projection features through a learnable gating mechanism to generate gated fused features. A top-down multi-scale fusion is performed on the gated fusion features. A dynamic weight modulation mechanism is introduced during the fusion process. After upsampling the high-level features, they are adaptively weighted and fused with the low-level features. P2-level shallow features from the backbone network are introduced, and after downsampling, they are adaptively weighted and fused with the same-level features from the top-down fusion process to obtain shallow enhanced features. Perform bottom-up multi-scale fusion on shallow enhancement features, downsample low-level features and adaptively weighted fuse them with high-level features to generate the final multi-scale fused feature map; The images of the logistics packages and waybills to be detected are input into the trained improved RT-DETR network model. The detection head network makes predictions based on the final multi-scale fusion feature map and outputs the detection results of the packages and waybills.

2. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 1, characterized in that, The process involves dynamically fusing the global context features obtained from modeling with the highest-level projected features using a learnable gating mechanism, specifically satisfying the following formula: in, The characteristics after fusion The global context features output by AIFI α represents the original projected features, and α is a learnable gating parameter.

3. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 1, characterized in that, The adaptive weighted fusion is implemented through an adaptive weighted feature pyramid module, which dynamically generates fusion weights based on the content of the input features. The calculation process follows this formula: in, For dynamic weights, For integrated output, For fully connected layer operations, This is a global average pooling operation. It is the Sigmoid activation function. For input features, This indicates multiplication by channel. It is a constant.

4. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 1, characterized in that, The process of introducing P2-level shallow features from the backbone network, after downsampling, and then adaptively weighting them with the same-level features from the top-down fusion process specifically includes: Extract P2-level high-resolution feature maps from the backbone network output; The number of channels in the P2-level high-resolution feature map is compressed to a preset uniform dimension through convolution operations; Perform a downsampling operation with a step size of 2 on the channel-compressed P2 level feature map to align its spatial resolution with that of the P3 level; The aligned features and the P3-level intermediate features generated during the top-down fusion process are input into the adaptive weighted feature pyramid module, and fusion is achieved through dynamic weight modulation.

5. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 1, characterized in that, After each execution of the adaptive weighted fusion, the improved neck network further includes a step of using the RepC3 module to process the fused features. The RepC3 module adopts a structure containing multi-branch convolutions during the training phase to enhance feature representation capabilities, and fuses the multi-branch convolutions into a single-path convolution through structural reparameterization technology during the inference phase to reduce inference computational overhead.

6. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 5, characterized in that, The feature processing of the RepC3 module during the training phase satisfies the following formula: Where X is the input feature. This indicates a block containing multiple reparameterized convolutional blocks, each consisting of... , Feature extraction branches consisting of identity branches; This indicates splicing along the channel dimension; finally, through... Perform feature fusion.

7. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to claim 1, characterized in that, The detection head network employs a decoder based on the Transformer architecture and iteratively optimizes the target query through a deformable attention mechanism, which satisfies the following formula: in, Target query features, For reference point location, For multi-scale feature maps, The number of sampling points. For the number of attention heads, For attention weights, For learnable sampling offset, For deformable attention functions, Let be the output projection weight matrix for the k-th sampling point. The projection weight matrix is ​​used to represent the value of the k-th sampling point.

8. The logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion according to any one of claims 1 to 7, characterized in that, The training process of the improved RT-DETR network model uses loss functions including the Focal Loss loss function for classification tasks and the GIoU loss function for regression tasks.

9. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion as described in any one of claims 1 to 8, and the processor is configured to execute the program stored in the memory.

10. A storage medium storing a computer program, characterized in that, When a computer program is run by a processor, it executes the steps of the logistics parcel and waybill detection method based on multi-scale feature bidirectional fusion as described in any one of claims 1 to 8.

Images